A blog about Code, the Web and Cyberculture.

Comparing Classifiers

info@martin-thoma.de (Martin Thoma) — Tue, 19 Jan 2016 20:13:00 +0100

Classification problems occur quite often and many different classification algorithms have been described and implemented. But what is the best algorithm for a given error function and dataset?

I read questions like “I have problem X. What is the best classifier?” quite often and my first impulse is always to write: Just try them!

I guess people asking this question might think that it is super difficult to do so. However, the sklearn tutorial contains a very nice example where many classifiers are compared (source).

This article gives you an overview over some classifiers:

Of course, neural networks are also one very powerful ML classifier I may not forget. As sklearn does not have neural networks, I’ve installed skflow.

Tutorial example

The sklearn tutorial creates three datasets with 100 points per dataset and 2 dimensions per point:

Moons: Two interleaving half-circles
Circles: A larger circle containing the smaller one
Linear: A linearly seperable dataset

Each of those three datasets has added noise. This means for some points there might be no way of classifying them correclty.

Here are the results

k nearest neighbors, linear and RBFSVM

One can see that k nearest neighbors gives arbitrary decision boundaries. Overall, they look reasonable. However, there are often strange zig-zag patterns.

The linear SVM in contrast has a very easy decision boundary: a line. It is no suprise that it can’t deal with the moons dataset. Note that a random guess would be right in 50% of the cases.

The RBF SVM has very nice decision boundary. It is smooth, matches the pattern and is able to adjust to all three examles.

Decision Tree, Random Forest, AdaBoost

Decision Trees, Decision Forests and AdaBoost all show very similar patterns. The boundaries change in parallel to the coordinate axes which looks very unnatural.

Naive Bayes, LDA, QDA

Naive Bayes shows nice, smooth patterns. However, those patterns seem to be a bit too simple. LDA is again linear (see linear SVM). Comparing QDA to Naive Bayes is interesting. Although they get similar performance for the first dataset, I would argue that the naive bayes classifier is much better as it is much more confident for its classification. Even more extrem is the last example. I’m astonished that the QDA gets 93% with that boundary; Naive Bayes seems to find a much better boundary.

The hardware

The following comparison is done on a PC with an Intel i7-4820K CPU and a NVIDIA GeForce GTX Titan Black GPU.

MNIST

MNIST is a dataset of $28\text{px} \times 28\text{px}$ greyscale images. Each of the images contains a digit (0, 1, 2, 3, 4, 5, 6, 7, 8, 9). The task is to classify the image into one of the 10 digit classes.

Guessing randomly will give an accuracy of $\frac{1}{10} = 0.1$.

Neural Networks

Please note that there are neural networks which get much better accuracy. Most notably the MNIST Expert tutorial with 99.2% accuracy.

Simple Network

Classifier: NN 500:200
Training time: 79.5696s
Testing time: 0.3480s
Confusion matrix:
[[2248    1    5    1    2    4    8    2    5    2]
 [   1 2565   10    1    1    0    2    7    1    0]
 [   7    2 2258   14    5    0    9    6   10    3]
 [   0    0   12 2294    0   23    0    6    3   10]
 [   0    3    3    0 2161    0    8    5    1   30]
 [   4    1    1   16    1 2014   17    1    5    9]
 [  11    7    1    0    5    6 2237    0    4    0]
 [   3    6   14    7    3    1    0 2355   10   18]
 [   3    7    3   14    2   17    4    1 2161    3]
 [   4    4    0    4   16    8    0    7    6 2340]]
Accuracy: 0.9798

Dropout Network

Classifier: NN 500:200 dropout
Training time: 118.2654s
Testing time: 0.3918s
Confusion matrix:
[[2250    1    7    1    1    1    5    4    4    4]
 [   1 2567    9    1    1    0    0    3    5    1]
 [   6    6 2272    3    2    1    3   10    8    3]
 [   0    0   26 2260    0   24    0   10   19    9]
 [   0    3    5    0 2152    0    7    3    1   40]
 [   8    3    3   12    2 1983   20    6   21   11]
 [  11    6    3    0    7    1 2237    0    6    0]
 [   2    7   13    3   11    0    1 2363    5   12]
 [   7    7    9    5    3    3    1    2 2170    8]
 [   3    3    1    3   13    2    0   19    8 2337]]
Accuracy: 0.9780

CNN

Classifier: CNN
Training time: 391.8810s
Testing time: 1.2035s
Confusion matrix:
[[2243    0    5    0    0    5    9    1   12    3]
 [   1 2548   20    4    2    0    1    6    6    0]
 [   3    8 2253    9    3    1    4   17   14    2]
 [   0    4   13 2290    0   12    0    9   11    9]
 [   2    4    5    0 2164    0    8    5    3   20]
 [   6    2    3   15    0 2016    9    3    9    6]
 [  12   12    1    1    6    6 2227    0    6    0]
 [   3    4   11    3    4    1    0 2374    4   13]
 [   3   15    6   13    4   11    3    8 2145    7]
 [   6    5    0   11   16    8    0   24   13 2306]]
Accuracy: 0.9769

SVM

There is a ton of literature / papers about SVMs. I’ve summed up the basics on Using SVMs with sklearn.

I’ve trained two SVMs: A simple, linear one and one with an RBF kernel as I found it online (I’m sorry, I don’t remember where I found those parameters :-/).

Please note the the SVM implementation of sklearn does not use the GPU. However, there are GPU implmentations of SVMs around.

Linear SVM

Classifier: linear SVM
Training time: 168.6950s
Testing time: 158.0101s
Confusion matrix:
[[2226    0    9    2    6   12    8    3   11    1]
 [   1 2537   18    3    3    1    1    7   17    0]
 [  12   16 2158   25   24    6   27   19   25    2]
 [   3    7   46 2188    4   47    3   18   27    5]
 [   2    5   19    1 2117    1    8    6    3   49]
 [  18   13   11   73   20 1872   31    0   26    5]
 [  20    6   22    1   10   30 2179    0    3    0]
 [   5   10   32   11   30    5    0 2268    5   51]
 [  11   39   26   47   10   40    7    7 2018   10]
 [  11    9    9   24   64    8    0   61   14 2189]]
Accuracy: 0.9416

Adjusted SVM

Classifier: adj. SVM
Training time: 347.1539s
Testing time: 234.5724s
Confusion matrix:
[[2258    1    4    1    2    2    3    1    4    2]
 [   1 2566    9    1    1    0    0    7    3    0]
 [   4    1 2280    5    4    0    1    9    8    2]
 [   0    0   14 2304    1   13    0    6    8    2]
 [   2    2    2    0 2183    0    7    5    0   10]
 [   4    0    0   16    3 2026   12    1    4    3]
 [   7    5    3    0    5    2 2245    0    4    0]
 [   1    6   11    2    5    1    0 2373    5   13]
 [   3    9    4    9    4   10    2    3 2166    5]
 [   3    2    2    6   19    6    0   12   10 2329]]
Accuracy: 0.9840

Random Forest

Data:

n_estimators=50
n_jobs=10

Classifier: Random Forest
Training time: 2.1359s
Testing time: 26.0763s
Confusion matrix:
[[2246    1    4    1    4    2    7    2   11    0]
 [   1 2543   18    5    5    2    3    7    4    0]
 [   7    2 2233   20    9    2    9   16   14    2]
 [   0    3   36 2240    0   20    3   16   19   11]
 [   3    1    5    0 2142    1   11    3    7   38]
 [   7    4    4   30    6 1977   16    3   14    8]
 [  13   11    4    0   10   15 2210    0    8    0]
 [   3    8   29    2   19    0    0 2315    7   34]
 [   3   12   18   17    9   26    4    7 2103   16]
 [  10    6    6   24   27   13    3   20   18 2262]]
Accuracy: 0.9641

Alternatively:

max_depth=5
n_estimators=10
max_features=1

Classifier: Random Forest 2
Training time: 0.2077s
Testing time: 22.2770s
Confusion matrix:
[[1955   32   63   64   12    4  109   21   13    5]
 [   1 2524   20   14    1    6   10    6    6    0]
 [ 252  425 1198  151   64    1  145   15   55    8]
 [ 136  195  140 1641   28   11   22   95   65   15]
 [  92  320   21   45 1199    9   76  153    8  288]
 [ 312  383   67  655   78  268   47   94  134   31]
 [ 199  364  125   58   96   13 1408    5    2    1]
 [  83  424   10   70  101    1   19 1555   56   98]
 [ 392  574   44  147   52   17   71  106  773   39]
 [  71  338   11   43  579    2    8  632   24  681]]
Accuracy: 0.5715

k nearest neightbors

Classifier: k nn
Training time: 4.6439s
Testing time: 1261.7815s
Confusion matrix:
[[2260    1    4    0    0    1    6    2    2    2]
 [   0 2572    5    0    0    0    1    8    1    1]
 [  16   15 2235    9    1    0    5   26    5    2]
 [   2    5   14 2276    0   27    1    8    9    6]
 [   4   19    0    0 2131    0    8    4    0   45]
 [  10    5    3   28    5 1977   25    2    4   10]
 [  12    9    0    0    4    7 2239    0    0    0]
 [   1   18    4    1   12    0    0 2349    3   29]
 [  11   32    8   36   11   34    5    7 2053   18]
 [   6    8    4   14   26    4    0   19    5 2303]]
Accuracy: 0.9695

Decision Tree

Data:

max_depth=5

Classifier: Decision Tree
Training time: 3.1346s
Testing time: 0.0313s
Confusion matrix:
[[1767    0   11   25   12  120  137   71  114   21]
 [   1 2065  128  108   13   17   41   66  131   18]
 [  42   44 1248   37  121   21  227   76  339  159]
 [  33   22   32 1484   33  107   52   81  266  238]
 [   0   15   45   33 1284   42   42   45  213  492]
 [  42   10   21  229  166  577  137  123  254  510]
 [  34   33   66   24  103   65 1734   24  102   86]
 [  10   14  179   57   53   21   19 1775   79  210]
 [   1   98  129   43   43   42  160   29 1439  231]
 [   4    8   86   59  125   95   36   75  167 1734]]
Accuracy: 0.6540

Adaboost

You should note that you can use arbitrary base classifiers with Adaboost. The default ones of sklearn.ensemble.AdaBoostClassifier is sklearn.tree.DecisionTreeClassifies

Classifier: AdaBoost
Training time: 37.6443s
Testing time: 1.5815s
Confusion matrix:
[[1994    0   75    8    6  113   51    3   15   13]
 [   0 2435   27   22    2   10   12   37   42    1]
 [  97   39 1341   85   39   38  416   39  196   24]
 [ 108   52   37 1508   13  313   66   64  122   65]
 [  11   16   48   23 1662   49   23  134   90  155]
 [  81   56   30  309   51 1255   57   17  129   84]
 [  29   28  151    7   80   43 1914    2   17    0]
 [  25   37   33   36   70   30    0 1761   37  388]
 [  30   80   48  215   16   85   30   19 1615   77]
 [  13   29   68   66  356   74    1  171   78 1533]]
Accuracy: 0.7367

Gradient Boosting

Gradient boosting with xgboost has won in the Rossmann Store Sales prediction (source).

See also:

Classifier: Gradient Boosting
Training time: 2409.8094s
Testing time: 0.4159s
Confusion matrix:
[[2214    1    3    5   10    8    9    3   24    1]
 [   1 2528   16   11    3    5    5    7    9    3]
 [   8    5 2165   34   16    5   12   22   37   10]
 [   1    9   27 2182    4   42    1   22   37   23]
 [   5    4   16    1 2088    5   12    5   10   65]
 [   9    6    7   41    8 1928   27    6   18   19]
 [  15    7    4    1   19   29 2181    1   14    0]
 [   6   16   27   15   22    6    0 2246    8   71]
 [   5   20   14   25   15   29    6    6 2057   38]
 [   6   10    8   24   49   15    1   54   17 2205]]
Accuracy: 0.9435

Naive Bayes

Classifier: Naive Bayes
Training time: 0.3814s
Testing time: 0.8863s
Confusion matrix:
[[2094    4   11   10    6    7   56    3   69   18]
 [   4 2432    9   11    2    4   28    1   77   20]
 [ 278   64  703  143    6    4  558    4  528   26]
 [ 202  136   18  791    5    5  106   21  886  178]
 [  96   26   16   14  296    8  169   13  535 1038]
 [ 327   63   15   39   14   87  100    5 1253  166]
 [  34   51   17    1    1    5 2109    0   52    1]
 [  19   21    3   23   20    2    7  737  123 1462]
 [  39  326   13   16    8   18   25    7 1482  281]
 [  15   26    8    2   14    2    1   41   40 2240]]
Accuracy: 0.5615

LDA

Classifier: LDA
Training time: 20.6464s
Testing time: 0.0910s
Confusion matrix:
[[2131    2   10   14   12   47   20    4   36    2]
 [   0 2454   20   10    5   16    5    5   71    2]
 [  22   71 1873   77   51    8   82   20  101    9]
 [   5   32   56 1992   11   77   11   40   80   44]
 [   1   21   17    0 1983   12   12    2   21  142]
 [  19   18   11  112   18 1682   37   11  103   58]
 [  28   30   32    3   43   51 2046    0   37    1]
 [  16   57   25   20   70    8    0 1990   11  220]
 [   9  113   16   64   33  115   13   10 1781   61]
 [  15   10    6   35  133   14    0  122   22 2032]]
Accuracy: 0.8642

QDA

Classifier: QDA
Training time: 23.0527s
Testing time: 6.2259s
Confusion matrix:
[[2212    3   12   14    1    4   20    5    6    1]
 [  66 2409   12   10    0    0   32    2   39   18]
 [ 961   25  689  143    3    1  310    2  166   14]
 [1231   48   29  606    3   13   66   10  232  110]
 [ 810   22   25   27  250    4  143   17  345  568]
 [ 909   15   13   33    1  214  140    4  666   74]
 [  83   18   14    1    1    2 2146    0    6    0]
 [  81   13    6   52   14    2    1  776  120 1352]
 [ 487  181   18   20    6   17   58    3 1320  105]
 [  65   14   12    7   10    0    0   23   33 2225]]
Accuracy: 0.5561

MNIST Summary

Classifier	Accuracy	Training Time	Testing Time
MLP (500:200)	97.98%	79.5696s	0.3480s
Dropout NN (500:200)	97.80%	118.2654s	0.3918s
CNN (32 5×5 filters : 2×2 max pool : 64 5×5 filters : 2×2 max pool : 1024)	97.69%	391.8810s	1.2035s
Adjusted SVM	98.40%	347.1539s	234.5724s
Linear SVM	94.16%	168.6950s	158.0101s
Random Forest (n_estimators=50, n_jobs=10)	96.41%	2.1359s	26.0763s
Random Forest (n_estimators=10, max_features=1, max_depth=5)	57.15%	0.2077s	22.2770s
k nearest neightbors (k=3)	96.95%	4.6439s	1261.7815s
Decision Tree(max_depth=5)	65.40%	3.1346s	0.0313s
Adaboost	73.67%	37.6443s	1.5815s
Naive Bayes	56.15%	0.3814s	0.8863s
LDA	86.42%	20.6464s	0.0910s
QDA	55.61%	23.0527s	6.2259s
Gradient Boosting	94.35%	2409.8094s	0.4159s
Logistic Regression (C=1)	91.47%	272.1309s	0.0531s
Logistic Regression (C=10000)	91.23%	1807.0624s	0.0529s

IRIS summary

Just for fun, I tried the script from above with very minor adjustments to the IRIS flower dataset:

Classifier	Accuracy	Training Time	Testing Time
AdaBoost	92.00%	0.1203s	0.0101s
Decision Tree	92.00%	0.0005s	0.0001s
Gradient Boosting	92.00%	0.2227s	0.0007s
LDA	96.00%	0.0027s	0.0002s
NN 20:5	90.00%	1.6628s	0.0046s
Naive Bayes	90.00%	0.0010s	0.0004s
QDA	94.00%	0.0009s	0.0003s
Random Forest	90.00%	0.2147s	0.1395s
Random Forest 2	90.00%	0.1481s	0.1249s
SVM, adj.	90.00%	0.0010s	0.0004s
SVM, linear	88.00%	0.0006s	0.0002s
k nn	92.00%	0.0007s	0.0009s
Logistic Regression (C=1)	88.00%	0.0011s	0.0001s
Logistic Regression (C=1000)	92.00%	0.0010s	0.0002s
RBM 100	78.00%	0.0233s	0.0003s
RBM 100, n_iter=20	70.00%	0.0427s	0.0003s
RBM 200, n_iter=40, LR=0.01, Reg: C=1	88.00%	0.2463s	0.0005s
RBM 200, n_iter=40, LR=0.01, Reg: C=10000	90.00%	0.2437s	0.0005s
RBM 256	84.00%	0.0424s	0.0006s
RBM 512, n_iter=100	84.00%	0.0723s	0.0010s

TL;DR

Neural networks take their time to train and a feeling for the topology, but their classification results are nice and the testing time is good as well.

Random Forests and SVMs are also a model a type of model one should think of. However, the stard implementation is very slow compared to neural networks.

sklearn.lda.LDA might also be worth a try. The rest seems to be quite bad compared with those classifiers.

The code which generated the examples from above is here.

Function Approximation

info@martin-thoma.de (Martin Thoma) — Mon, 18 Jan 2016 20:00:00 +0100

I was recently quite disappointed by how bad neural networks are for function approximation (see How should a neural network for unbound function approximation be structured?). However, I’ve just found that Gaussian processes are great for function approximation!

There are two important types of function approximation:

Interpolation: What values does the function have in between of known values?
Extrapolation: What values does the function have outsive of the known values?

I did a couple of very quick examples which look promising.

Examples

Square

Approximating $f(x) = x^2$ worked very good:

f(x) = x^2

I’ve tried if with higher order polynomials, more complex polynomials. No problem.

Sin

Approximating $f(x) = \sin(3x)$ seems to be more complicated:

f(x) = sin(3x)

I guess a human would see the wave pattern and do a better job here.

exp

Approximating $f(x) = e^x$ works similar well as polynomials. One can see that it does not perfectly fit it, but compared the the range of values seen before and the distance from the last seen value I think this is absolutely acceptable:

f(x) = e^x

noise

It is claimed that Gaussian processes implicitly model noise so that they can easily deal with noise. However, in my experients this seems not to work so great. The reason might be that I had points in $[-3, 3]$ of the function

\[f(x) = x^2\]

with point-wise gaussian noise $N \sim \mathcal{N}(0, 1)$. So the noise is quite domintant on that intervall. One of the examples where it worked better is:

f(x) = x^2 with gaussian noise

Make it brake

I was a bit suspicious if I had another mistake here. So I wanted it to break. This was the reason why I created the following function

\[f(x) = \begin{cases}x^2 &\text{if } x \geq 0\\-1 &\text{otherwise}\end{cases}\]

Function with discontinuity

The predicted value is obviously not correct, but you should note that almost all function values are within the 95% confidence intervall!

Code

The following code needs numpy and sklearn. For the plots, you need matplotlib.

#!/usr/bin/env python

"""Example how to use gaussion processes for regression."""

import numpy as np
from sklearn import gaussian_process


def main():
    # Create the dataset
    x_train = np.atleast_2d(np.linspace(-3, 3, num=50)).T
    y_train = f(x_train).ravel()
    x_test = np.atleast_2d(np.linspace(-5, 5, 1000)).T

    # Define the Regression Modell and fit it
    gp = gaussian_process.GaussianProcess(theta0=1e-2,
                                          thetaL=1e-4,
                                          thetaU=1e-3)
    gp.fit(x_train, y_train)

    # Evaluate the result
    y_pred, mse = gp.predict(x_test, eval_MSE=True)
    print("MSE: %0.4f" % sum(mse))
    print("max MSE: %0.4f" % max(mse))
    plot_graph(x_test, x_train, y_pred, mse, "x^2")


def f(x):
    """
    Function which gets approximated
    """

    noise = [np.random.normal(loc=0.0, scale=1.0) for _ in range(len(list(x)))]
    noise = np.atleast_2d(noise).T
    return x**2 + noise
    # Totally fails for that one:
    # y = []
    # for el in x:
    #     if el >= 0:
    #         y.append(el**2)
    #     else:
    #         y.append(-1)
    # return np.array(y)


def plot_graph(x, x_train, y_pred, mse, function_tex):
    # Plot the function, the prediction and the 95% confidence interval
    # based on the MSE
    sigma = np.sqrt(mse)
    from matplotlib import pyplot as pl
    pl.figure()
    y = f(x_train).ravel()
    pl.plot(x, f(x), 'r:', label=u'$f(x) = %s$' % function_tex)
    pl.plot(x_train, y, 'r.', markersize=10, label=u'Observations')
    pl.plot(x, y_pred, 'b-', label=u'Prediction')
    pl.fill(np.concatenate([x, x[::-1]]),
            np.concatenate([y_pred - 1.9600 * sigma,
                           (y_pred + 1.9600 * sigma)[::-1]]),
            alpha=.5, fc='b', ec='None', label='95% confidence interval')
    pl.xlabel('$x$')
    pl.ylabel('$f(x)$')
    y_min = min(min(y_pred), min(y)) * 1.1
    y_max = max(max(y_pred), max(y)) * 1.1
    pl.ylim(y_min, y_max)
    pl.legend(loc='upper left')
    pl.show()


if __name__ == '__main__':
    main()

Using SVMs with sklearn

info@martin-thoma.de (Martin Thoma) — Thu, 14 Jan 2016 12:25:00 +0100

Support Vector Machines (SVMs) is a group of powerful classifiers. In this article, I will give a short impression of how they work. I continue with an example how to use SVMs with sklearn.

SVM theory

SVMs can be described with 5 ideas in mind:

Linear, binary classifiers: If data is linearly separable, it can be separated by a hyperplane. There is one hyperplane which maximizes the distance to the next datapoints (support vectors). This hyperplane should be taken:

\[ \begin{aligned} \text{minimize}_{\mathbf{w}, b}\,&\frac{1}{2} \|\mathbf{w}\|^2\\ \text{s.t. }& \forall_{i=1}^m y_i \cdot \underbrace{(\langle \mathbf{w}, \mathbf{x}_i\rangle + b)}_{\text{Classification}} \geq 1 \end{aligned}\]
Slack variables: Even if the underlying process which generates the features for the two classes is linearly separable, noise can make the data not separable. The introduction of slack variables to relax the requirement of linear separability solves this problem. The trade-off between accepting some errors and a more complex model is weighted by a parameter $C \in \mathbb{R}_0^+$. The bigger $C$, the more errors are accepted. The new optimization problem is: \[ \begin{aligned} \text{minimize}_{\mathbf{w}}\,&\frac{1}{2} \|\mathbf{w}\|^2 + C \cdot \sum_{i=1}^m \xi_i\\ \text{s.t. }& \forall_{i=1}^m y_i \cdot (\langle \mathbf{w}, \mathbf{x}_i\rangle + b) \geq 1 - \xi_i \end{aligned}\] Note that $0 \le \xi_i \le 1$ means that the data point is within the margin, whereas $\xi_i \ge 1$ means it is misclassified. An SVM with $C > 0$ is also called a soft-margin SVM.
Dual Problem: The primal problem is to find the normal vector $\mathbf{w}$ and the bias $b$. The dual problem is to express $\mathbf{w}$ as a linear combination of the training data $\mathbf{x}_i$: \[\mathbf{w} = \sum_{i=1}^m \alpha_i y_i \mathbf{x}_i\] where $y_i \in \{-1, 1\}$ represents the class of the training example and $\alpha_i$ are Lagrange multipliers. The usage of Lagrange multipliers is explained with some examples in [Smi04]. The usage of the Lagrange multipliers $\alpha_i$ changes the optimization problem depend on the $\alpha_i$ which are weights for the feature vectors. It turns out that most $\alpha_i$ will be zero. The non-zero weighted vectors are called support vectors. The optimization problem is now, according to [Bur98] (a great read; if you really want to understand it I can recommend it!): \[ \begin{aligned} \text{maximize}_{\mathbf{w}}\,& \sum_{i=1}^m \alpha_i - \frac{1}{2} \sum_{i=1}^m \sum_{j=1}^m \alpha_i \alpha_j y_i y_j \langle \mathbf{x}_i, \mathbf{x}_j \rangle\\ \text{s.t. } & \forall_{i=1}^m 0 \leq \alpha_i \leq C\\ \text{s.t. } & \sum_{i=1}^m \alpha_i y_i = 0 \end{aligned}\]
Kernel-Trick: Not every dataset is linearly separable. This problem is approached by transforming the feature vectors $\mathbf{x}$ with a non-linear mapping $\Phi$ into a higher dimensional (probably $\infty$-dimensional) space. As the feature vectors $\mathbf{x}$ are only used within scalar product $\langle \mathbf{x}_i, \mathbf{x}_j \rangle$, it is not necessary to do the transformation. It is enough to do the calculation \[K(\mathbf{x}_i, \mathbf{x}_j) = \langle \mathbf{x}_i, \mathbf{x}_j \rangle\] This function $K$ is called a kernel. The idea of never explicitly transforming the vectors $\mathbf{x}_i$ to the higher dimensional space is called the kernel trick. Common kernels include the polynomial kernel \[K_P(\mathbf{x}_i, \mathbf{x}_j) = (\langle \mathbf{x}_i, \mathbf{x}_j \rangle + r)^p\] of degree $p$ and coefficient $r$, the Gaussian RBF kernel \[K_{\text{Gauss}}(\mathbf{x}_i, \mathbf{x}_j) = e^{\frac{-\gamma\|\mathbf{x}_i - \mathbf{x}_j\|^2}{2 \sigma^2}}\] and the sigmoid kernel \[K_{\text{tanh}}(\mathbf{x}_i, \mathbf{x}_j) = \tanh(\gamma \langle \mathbf{x}_i, \mathbf{x}_j \rangle - r)\] where the parameter $\gamma$ determines how much influence single training examples have.
Multiple Classes: By using the one-vs-all or the one-vs-one strategy it is possible to get a classifying system which can distinguish many classes.

A nice visualization of the transformation of the data in a higher-dimensional space was done by

TeamGrizzly’s channel: Performing nonlinear classification via linear separation in higher dimensional space on YouTube. 22.11.2010.

See also:

sklearn

sklearn is the machine learning toolkit to get started for Python. It has a very good documentation and many functions. You can find installation instructions on their website.

It also includes sklearn.svm.SVC. SVC is short for support vector classifier and this is how you use it for the MNIST dataset.

Parameters for which you might want a further explanation:

cache_size: datascience.stackexchange.com

#!/usr/bin/env python

"""
Train a SVM to categorize 28x28 pixel images into digits (MNIST dataset).
"""

import numpy as np


def main():
    """Orchestrate the retrival of data, training and testing."""
    data = get_data()

    # Get classifier
    from sklearn.svm import SVC
    clf = SVC(probability=False,  # cache_size=200,
              kernel="rbf", C=2.8, gamma=.0073)

    print("Start fitting. This may take a while")

    # take all of it - make that number lower for experiments
    examples = len(data['train']['X'])
    clf.fit(data['train']['X'][:examples], data['train']['y'][:examples])

    analyze(clf, data)


def analyze(clf, data):
    """
    Analyze how well a classifier performs on data.

    Parameters
    ----------
    clf : classifier object
    data : dict
    """
    # Get confusion matrix
    from sklearn import metrics
    predicted = clf.predict(data['test']['X'])
    print("Confusion matrix:\n%s" %
          metrics.confusion_matrix(data['test']['y'],
                                   predicted))
    print("Accuracy: %0.4f" % metrics.accuracy_score(data['test']['y'],
                                                     predicted))

    # Print example
    try_id = 1
    out = clf.predict(data['test']['X'][try_id])  # clf.predict_proba
    print("out: %s" % out)
    size = int(len(data['test']['X'][try_id])**(0.5))
    view_image(data['test']['X'][try_id].reshape((size, size)),
               data['test']['y'][try_id])


def view_image(image, label=""):
    """
    View a single image.

    Parameters
    ----------
    image : numpy array
        Make sure this is of the shape you want.
    label : str
    """
    from matplotlib.pyplot import show, imshow, cm
    print("Label: %s" % label)
    imshow(image, cmap=cm.gray)
    show()


def get_data():
    """
    Get data ready to learn with.

    Returns
    -------
    dict
    """
    simple = False
    if simple:  # Load the simple, but similar digits dataset
        from sklearn.datasets import load_digits
        from sklearn.utils import shuffle
        digits = load_digits()
        x = [np.array(el).flatten() for el in digits.images]
        y = digits.target

        # Scale data to [-1, 1] - This is of mayor importance!!!
        # In this case, I know the range and thus I can (and should) scale
        # manually. However, this might not always be the case.
        # Then try sklearn.preprocessing.MinMaxScaler or
        # sklearn.preprocessing.StandardScaler
        x = x/255.0*2 - 1

        x, y = shuffle(x, y, random_state=0)

        from sklearn.cross_validation import train_test_split
        x_train, x_test, y_train, y_test = train_test_split(x, y,
                                                            test_size=0.33,
                                                            random_state=42)
        data = {'train': {'X': x_train,
                          'y': y_train},
                'test': {'X': x_test,
                         'y': y_test}}
    else:  # Load the original dataset
        from sklearn.datasets import fetch_mldata
        from sklearn.utils import shuffle
        mnist = fetch_mldata('MNIST original')

        x = mnist.data
        y = mnist.target

        # Scale data to [-1, 1] - This is of mayor importance!!!
        x = x/255.0*2 - 1

        x, y = shuffle(x, y, random_state=0)

        from sklearn.cross_validation import train_test_split
        x_train, x_test, y_train, y_test = train_test_split(x, y,
                                                            test_size=0.33,
                                                            random_state=42)
        data = {'train': {'X': x_train,
                          'y': y_train},
                'test': {'X': x_test,
                         'y': y_test}}
    return data


if __name__ == '__main__':
    main()

Results

The script from above gives the following results:

Confusion matrix for an SVM classifier on the MNIST dataset
	0	1	2	3	4	5	6	7	8	9
0	2258	1	4	1	2	2	3	1	4	2
1	1	2566	9	1	1	0	0	7	3	0
2	4	1	2280	5	4	0	1	9	8	2
3	0	0	14	2304	1	13	0	6	8	2
4	2	2	2	0	2183	0	7	5	0	10
5	4	0	0	16	3	2026	12	1	4	3
6	7	5	3	0	5	2	2245	0	4	0
7	1	6	11	2	5	1	0	2373	5	13
8	3	9	4	9	4	10	2	3	2166	5
9	3	2	2	6	19	6	0	12	10	2329

Accuracy: 98.40%
Error: 1.60%

Looks pretty good to me. However, note that there are much better results. The best on the official website has an error of 0.23% and is a committee of 35 convolutional neural networks.

The best SVM I could find has an error of 0.56% and applies a polynomial kernel of degree 9 as well as some preprocessing.

References

[Smi04] B. T. Smith, “Lagrange multipliers tutorial in the context of support vector machines,” Memorial University of Newfoundland St. John’s, Newfoundland, Canada, Jun. 2004.
[Bur98] C. J. Burges, “A tutorial on support vector machines for pattern recognition”, Data mining and knowledge discovery, vol. 2, no. 2, pp. 121–167, 1998.

Explaining Away

info@martin-thoma.de (Martin Thoma) — Mon, 04 Jan 2016 23:52:00 +0100

Explaining away is an effect where which is explained in Pearl (1988) with an example similar to the following one:

A car’s engine can fail ($X$). The reason might either be a dead battery $Y$ or a blocked fuel pump $Z$.

This results in the following Bayesian Network:

A common effect

Now assume you know that the engine does not fail ($X=0$). This guarantees that the battery is not dead ($Y=0$) and the fuel pump is not blocked ($Z=0$).

However, if the engine failed ($X=1$) it gets more interesting. Let’s say the engine works with a probability of 90%. From the graph, you can see that we assume that a dead battery is independant of a blocked fuel pump. Let’s also assume that you know that either the fuel pump is blocked or the battery is dead when the engine failed. There is no other option.

This makes setting the probability distribution up interesting. There is still 10% left which has to be distributed amongst $P(X=1,Y=1,Z=1), P(X=1,Y=1,Z=0), P(X=1,Y=0,Z=1)$:

X	Y	Z	P(X,Y,Z)	comment
0	0	0	0.9	Engine works
0	0	1	0	impossible - blocked fuel pump and working engine
0	1	0	0	impossible - dead battery and working engine
0	1	1	0	impossible - dead battery, blocked fuel pump and working engine
1	0	0	0	impossible - a cause we don’t have in our graph
1	0	1	a
1	1	0	b
1	1	1	c

However, if you assume that $Y$ and $Z$ are independent, given $X$, then you would have to set $P(Y=0, Z=0|X=1) = P(Y=0|X=1) \cdot P(Z=0|X=1)$ which would not be 0 except one of the probabilities would be 0. Or, putting it different again: Given that you know the engine is broken, the event of a dead battery and a blocked fuel pump are suddenly not independent anymore!

What does it mean

This does not mean there is “suddenly” a causatal connection between a dead battery and a blocked fuel pump. It only means you can learn something for your predictions.

How to clear a USB stick

info@martin-thoma.de (Martin Thoma) — Sat, 02 Jan 2016 22:57:00 +0100

Once in a while I think it is time to reduce the damage being done by the loss of a USB stick.

USB sticks

Remove all data

Find the USB stick on your Linux system with fdisk -l. Make super sure that you really got the stick (e.g. by removing it and executing the command again). It should be something like /dev/sdb1.

The following command will overwrite the data on the stick 5 times:

$ shred -f -n 5 /dev/sdX

This may take several minutes even for very small USB sticks.

Format the stick

Start gparted and use the GUI.

Add README

In case I lose the stick, adding a README.txt makes with a way to contact me makes it most likely that I get the stick back.

Dieser USB-Stick ging verloren.

Mein Name ist Martin Thoma. Sie können mich per E-Mail unter
info@martin-thoma.de erreichen.

---

This USB-Stick was lost.

My name is Martin Thoma. You can contact me via e-mail: info@martin-thoma.de

Preparing for the New Year

info@martin-thoma.de (Martin Thoma) — Thu, 31 Dec 2015 14:37:00 +0100

It’s new years eve and - as always - I try to finish some things and have some plans for next year.

Review of 2015

The last year has had it ups and downs. Most of them are private, so I’ll not write too much about it. However, I would like to share two of them with you: I’ve been on my first summer school of the German National Academic Foundation (Sommerakademie der Studienstiftung, see photos) and I’ve been for two weeks hiking the west highland way (see photos). This month was really awesome.

Here are some other mentionable facts:

I’ve founded Machine Learning Karlsruhe. See ml-ka.de for more information.
I’ve got 10 000 points on StackOverflow and reached 2.5 million people with my questions and answers (source)
I’ve got the funding by the Begabtenstiftung to go with my friend Marvin to Berkeley. Let’s see if Berkeley accepts us.
I’ve published 4 papers on arXiv (link) (one is still on hold) and started building an academic profile with orcid.org/0000-0002-6517-1690. I’m not sure how important that will be, though.
I posted 36 articles on my blog in 2015. However, I don’t think there is a single good article amongst them. I simply didn’t have enough time to write an interesting article ☹. There are 47 drafts from 2015; 134 drafts in total.
write-math.com now has
- 2520 users who contributed something
- 1595 formulas (that includes simple symbols)
- 274244 recordings. 6495 of those are not classified yet.

Finishing Things

I always do some cleanup before the new year begins. This means especially cleaning my computer stuff of unnecessary trash.

E-Mail

Finding and deleting old unnecessary emails:

StackExchange: do-not-reply@stackexchange.com
University
- physik-l@lists.kit.edu
- physik-ll@lists.kit.edu
- stipendiaten@lists.does-not-exist.org
- ilias@scc-ilias-03.scc.kit.edu
Programming
- notifications@travis-ci.org
- noreply@coursera.org
Other
- notify@twitter.com
- jobs-listings@linkedin.com
- messages-noreply@linkedin.com
- noreply@indiegogo.com
- mailing@abgeordnetenwatch.de
- offener-haushalt-request@lists.okfn.org
- infoen@bondora.com

This were several hundret emails.

The next step is going through “marked emails”.

Desktop and Downloads

Clean those folders up … *sigh*… I didn’t get ready.

Updating

# apt-get update
# apt-get upgrade
$ upgrade_oh_my_zsh

Plans

I want to get the following things done in 2016:

[ ] Quit church: That one is actually more complicated in Germany than it should be. I have to get to the civil registry office, get a personal date, pay 31.50 Euro (Source: karlsruhe.de)
[ ] Migrate from Jekyll to Pelican (see article draft)
[ ] Get all my stuff from my fathers apartment to my room. I’ve already packed most of it in boxes (In total: 1.30m (+0.30m?) × 0.39m × 0.60m by 7 boxes)
[ ] Get rid of my old bed
[ ] Get rid of my old vacuum cleaner
[ ] Paint the wall
[ ] Contact BSI, BKA, CCC and the car sellers and send them my paper in which I summarize current problems in car security.
[ ] Getting new slippers

Of course, I have a lot of other plans. Although some people say I don’t do anything besides computer stuff, I do have a prive life. But in contrast to others I like to keep it private.

I wish all of you a happy new year!

Predictions for 2016

I’ve just seen Predictions for 2016 on Computational Complexity and thought it is a fun idea. It is currently the 4th of January, so the year is still young enough to not consider this cheating:

Machine Learning and Technology

Networks will get deeper. Currently about 8 layers is usual; I expect this to get to over 30 layers.
First hacks / funny results by people abusing the fact that machine learning is used will appear.
Machines will generate “high” resolution images of at least 256 &times 256 pixels.
Speech synthesis will fastly improve. To benchmark this, I will use the original quotation that explained the importance of towels is found in Chapter 3 of Adams’ work The Hitchhiker’s Guide to the Galaxy. See Towel Day, Google Translate Recording, Festival Recording
ALPHABET INC. (C) (GOOGLE) shares at Frankfurt Stock Exchange will go up to about 900 Euro sometime in the year (currently, it is at 690 Euro).
Solar Roadways will make some tiny advances.

About Me

One of my papers will get cited at least 5 times.
I will spend at least 2 weeks abroad.
I will publish at leat 40 articles on my blog.
I will get a 90 days streak on GitHub (currently, my longest is 43 days).
I will get 15 000 points on StackExchange (see StackExchange profile)

Politics

The European Union will still exist and Britain will still be part of it (see Brexit).
Donald Trump will be the republican candidate and Hillary Clinton the democrat candidate.
The republican candidate will get president.
More mass shootings in the US and terrorist attacks all over the world will happen.
Grüne and SPD will still be in Baden-Württemberg (see Landtagswahl in Baden-Württemberg 2016).

Analyzing PyPI Data - 2

info@martin-thoma.de (Martin Thoma) — Wed, 30 Dec 2015 13:28:00 +0100

This is part two of a series. See Analyzing PyPI Data for part one.

I’ve recently got a request to expand my analysis of the Python Package Index commonly known as PyPI. It is a repository of Python packages where everybody can upload packages; pretty much without any restriction. In the article Analyzing PyPI Metadata you can read some general stuff about the repository.

This article is going a bit more deeper. This time I don’t only analyze the metadata, but the relationship of the packages themselves. I wanted to build a dependency graph. However, here is a downside of Pythons package structure: The file which defines the dependencies of a Python package is a Python script itself. This gives the package developer the highest flexibility, but it also gives them the power to execute arbitrary code when I only want to get the dependencies.

As I am pretty sure there are some malicious packages in the repository (Although I’ve never heard of a single one there has to be one. Over 50 000 packages by 2015 - there has to be one!). So I don’t want to execute any code of the repository without having at least a clue what it should do. This means my analysis is very simple and thus prone to some errors.

Most common single dependency

One can see dependencies as being weighted by the number of times a package imports the package.

Non-weighted

SELECT
    `packages`.`name`,
    COUNT(`needs_package`)
FROM
    `dependencies`
JOIN
    `packages` ON `needs_package` = `packages`.`id`
GROUP BY
    `needs_package`
ORDER BY
    COUNT(`needs_package`) DESC
LIMIT 20

which gives

This bar chart displays which Python modules get imported by most Python packages

and without the system packages:

SELECT
    `packages`.`name`,
    COUNT(`needs_package`)
FROM
    `dependencies`
JOIN
    `packages` ON `needs_package` = `packages`.`id`
WHERE
    `on_pypi` = 1
GROUP BY
    `needs_package`
ORDER BY
    COUNT(`needs_package`) DESC
LIMIT 20

which gives

This bar chart displays which Python modules (excluding system modules) get imported by most Python packages

Weighted

How often gets a single module included over all packages?

SELECT
    `packages`.`name`,
    SUM(`times`)
FROM
    `dependencies`
JOIN
    `packages` ON `needs_package` = `packages`.`id`
GROUP BY
    `needs_package`
ORDER BY
    SUM(`times`) DESC
LIMIT 20

2 seconds later I’ve got the result:

Number of imports of Python packages

If I’m only interested in the packages which are on PyPI, hence not system packages, I execute the following query:

SELECT
    `packages`.`name`,
    SUM(`times`)
FROM
    `dependencies`
JOIN
    `packages` ON `needs_package` = `packages`.`id`
WHERE
    `on_pypi` = 1
GROUP BY
    `needs_package`
ORDER BY
    SUM(`times`) DESC
LIMIT 20

which gives me about 2 seconds later the following result:

Number of imports of Python packages, excluding system packages

Non-functional packages

Although there are many packages for Python which are very useful, there are also quite a lot which are not usefull at all. One possibility to identify such packages is by checking which packages get neither used by others nor use other packages

SELECT
    `packages`.`id`, `name`
FROM `packages`
WHERE
    `id` in (
        SELECT DISTINCT
            `dependencies`.`package`
        FROM
            dependencies)
OR `id` in (
        SELECT DISTINCT
            `dependencies`.`needs_package`
        FROM
            `dependencies`)

This leads to the result that 54 900 packages of 67 582 packages are not obviously crap. Or to write it in another way: 11 682 packages are crap. That is 17.5 %. Too much, in my opinion. However, this might also be due to my crappy script not downloading / checking the downloaded files correctly.

Names

One thing I was interested in while downloading all those packages was the question if there are malicious packages (either on purpose or by accident). One undesirable thing that could happen would be very similar names.

Prefixes

Let’s see how many packages are prefixes of other packages. My thought was that this might be developers trying to get some accidential installs. However, it only showed some relationships. I wanted to make a Levensthein distance analysis, but I guess this is not worth it.

Here are the top 10 strings which are prefixes of packages and packages themselves:

djan: 6462
pyt: 1626
pyth: 1278
collect: 1155
Flask: 561
open: 474
pyr: 442
pyp: 295
pyra: 285
pym: 256

One interesting thing I’ve learned is that you can use pip like this:

$ pip search "djan$"

… and I found a pip bug (github.com/pypa/pip/issues/3327)

Graph analysis

Analyzing the dependency graph is quite a challenge. Or at least that was what I initiallly thought. This graph has about 67 582 nodes and 436 980 edges. Quite a bit. Definitely much larger than what I have previously used.

However, my friend Nilan who knows a lot about graphs send me a link to StackOverflow: Visualizing Undirected Graph That’s Too Large for GraphViz?

This lead me to Gephi and the OpenOrd layout plugin. It didn’t work for the complete graph (see issues/1207), it worked after I removed the single nodes without edges.

Now we can ask several standard questions about graphs:

How many connected components are there?
Are there any circles? (That would be bad… dependency graphs should not have circles. Similar to family trees.)
Which are the most central nodes?

I didn’t find the time to answer those, but I put the graph data in JSON format on github.com/MartinThoma/pypi-dependencies. Please let me know when you do something interesting with the data.

I’ve only got some crappy images with Gephi / GraphViz:

Code

See github.com/MartinThoma/algorithms.

What could come next

I would like too measure the overall code quality on PyPI in another post. I think of the following measures:

pyroma: A 10-point score for packages
package goodness with Cheesecake,
pylint
PEP8 conformance,
Lines of code (LOC) / documentation / whitespace
Docstring style (None, NumpyDoc, Sphinx, Google - see Python Code Documentation)
Usage of functions
Testing coverage
Look for URLs in the code and which are reachable / which are not
Look for non-Python files

How to find new Papers

info@martin-thoma.de (Martin Thoma) — Thu, 03 Dec 2015 07:30:00 +0100

A couple of people who are just participating in their first seminar might ask themselves how you can find interesting new publications (papers). I would like to shed some light on it.

Initial Research

The hardest part is when you’re new to a field. You don’t know which journals are the important ones, which authors are most respected in the field, which terms are used.

Journal Rankings

Automated journal rankings like the h5-index used by Google Scholar (see metrics, Wikipedia) is one way to deal with the lack of domain knowledge. Google Scholar even publishes a list of journals per field (see Top publications in Computer Vision and Pattern Recognition).

Google Scholar Journal ranking

Algorithmic Ideas

I’ve never tried it (on purpose), but the idea just came to my head: Use a random walk. You start with some paper of which you don’t really know if it is good. Look at the references and try to weild out the ones which are obviously not work trying (e.g. because they are websites). Now take a random one to continue with. Do this a couple of times.

The idea is that papers which are more important are much more often cited. So you can hope to get to more important papers over time.

The hard way

Search for papers via Google. Look at the references. Search for papers in the reference. Get a feeling for the language being used.

Ongoing Research

When you have a little bit of domain knowledge (or a starting paper given to you by your supervisor) everything gets a lot easier.

There are a couple of key strategies:

Looking at the references of a paper
Looking at what else the same author published
Looking for similar titles
Reading Journals

For all three of them, I can highly recommend Google Scholar. They also offer a nice overview / statistics about authors so that you can get a feeling for their scientific activity. But don’t let you fool you by those numbers: It is possible to fake them.

The other nice thing is that Google Scholar makes citing publications incredibly easy:

Searching for a publication on Google Scholar

Citing a publication via Google Scholar

The arXiv

The arXiv can be seen as THE “modern way” journal. Depending on the field, it has a VERY high reputation.

For Pattern Recognition the reputation of the arXiv is amongst the highest ten journals, but it there are some fields like physics where it is THE top journal in several subfields like cosmology or high energy physics.

What makes the arXiv stand out is the fact that you can read new papers every day. I like to have a view at cs.CV/recent to keep myself informed about new stuff. There are about 20 new publications per day. On some days, there is nothing interesting, on other days like today quite a bit:

Latest submissions on arXiv in Pattern Recognition and Computer Vision

Filtering

I filter publications like this:

First, I look at the title. If that doesn’t look interesting, I skip it. The more exact the title is, the better. If it is too vague, I sometimes skip it too because the authors seem not to be familiar with the scientific style of writing and thus are probably not able to write an interesting paper.
Then I read the abstract. I expect to get to know what the paper is about.
Then I look at images / tables (depending on the subject)
I read the conclusion / last part
I read the end of the introduction where it often says “the paper is structured as follows”

I guess many researchers filter like this. This is important to know when you write a paper.

OpenBCI

info@martin-thoma.de (Martin Thoma) — Thu, 26 Nov 2015 22:40:00 +0100

I’ve just discovered the OpenBCI: Biosensing for Everybody Kickstarter project. BCI is short for Brain Computer Interface. This Kickstarter is about an EEG device. Such a device looks like a helmet. It has several electrodes which you place on your scalp. Those electrodes measure “voltage fluctuations resulting from ionic current within the neurons of the brain” (source: Wikpedia).

I’m really curious about this, because I recently had some thoughts that medicine is currently not working as I want it to be. Doctors rely on the information they get by their patients, and they shouldn’t. Patients can either directly lie or say something which is not try by accident. Instead of taking this feedback from patients, decisions should more often rely on hard data.

For locked-in people, that means people who have sever difficulties to communicate with the outside world, it would be awesome if we could use BCIs to help them. I know this is far in the future, but you have to start at some point.

Another use I would really be curious is some kind of “automatic interrogation”. Think of terrorists / criminals who got caught, but are not willing to give the police necessary information. I think we should have the possibility to extract this kind of information from them, without hurting them. This idea was mentioned in one of the books of Daniel Suarez; I think it was Daemon.

The last nice application of BCIs would be REALLY convenient ways to interact with computers. Again, this is something really far in the future, but an awesome idea ☺

The Perks

Ganglion

It is an EEG board. Whatever that means:

The OpenBCI Ganglion is a high-quality, affordable bio-sensing device. On the input side, there are 4 high-impedance differential inputs, a driven ground (DRL), a positive voltage supply (Vdd), and a negative voltage supply (Vss). The inputs can be used as individual differential inputs for measuring EMG or ECG, or they can be individually connected to a reference electrode for measuring EEG.

Ultracortex Mark IV

The Ultracortex Mark IV is an EEG headset.

The Ultracortex Mark III is the latest working version of the OpenBCI headset. You can find all of the 3D files. links to hardware, and an assembly on our Github repo. We designed it for maximum adjustability and ease of use. In our design thinking, we prioritized the use of dry electrodes (pictured in the images above). Using dry sensors significantly reduces the time needed for setup (no more sticky paste!) and makes the overall experience of wearing the headset much more pleasant.

There are three options:

Print-It-Yourself (`$350): this reward comes with all of the pieces required to assemble a full Ultracortex Mark IV aside from the pieces you can print yourself with a desktop 3D-printer, in addition to an OpenBCI Ganglion Board. This kit is perfect for backers who have their own 3D-printer.

Unassembled ($`450): this reward comes with all of the pieces required to assemble a full Ultracortex Mark IV headset, including the pieces that are 3D-printable, and an OpenBCI Ganglion Board.

Fully Assembled ($650): if you select this reward, you will receive a fully- assembled Ultracortex Mark IV with an OpenBCI Ganglion Board. It will arrive ready to plug in and fire up!

What I would expect from it to be able to differentiate at least five different kinds of signals which I can willently make. Why five?

Up
Down
Right (Yes)
Left (No)
Enter

With those, you can make a virtual keyboard which is usable. Of course, you can also do so with two actions (next, enter), but that is MUCH more of a pain. The more different signals you can distinguish, the better.

I also expect it to be able to see when I feel pain or when I sleep.

Specs

Currently, there are basically no specs available (see comments). Some comments seem to suggest that there are only 16 electrodes in use. Then they speak about a 10-20 and a 10-10 system. I have no idea what that means.

The People behind it

The project is done by Joel Murphy and Conor Russomanno. They seem to have made a similar Kickstarter called “Spiderclaw”, but I could only find the GitHub repository. They also said they are affiliated with Pulse Sensor

In a video, Paul Sajda (PhD, Professor of Biomedical Engineering and Radiology at Columbia University, source) says it is a “high quality recording system”.

Ashley E. Stewart (CSO of Neuromore, see also brainbodyperformance.com)

Aaron Trocola (Industrial Designer at Threeform) seems to be the person who designs the 3D printed parts.

David Putrino, Department of Rehabilitation Medicine Weill Cornell Medical College (source) also said a couple of positive words about it.

Conclusion

By now, this project does not give me the impression that it is ready-to-use. I am still waiting for some feedback, but I guess I will rather wait for other (independant) people writing reviews about it. Preverably mentioning it in scientific papers which get published in notable journals or cited by enough other people.

Articles

I couldn’t find any articles about OpenBCI on scholar.google.com, except for the following:

Chip Audette, “Mind Control”, IEEE Spectrum. October 2015. DOI 10.1109/MSPEC.2015.7274184. Also: eeghacker.blogspot.de

Terror in Paris

info@martin-thoma.de (Martin Thoma) — Sun, 15 Nov 2015 10:29:00 +0100

Recently, a huge terrorist attack happened in France (see Wikipedia). At least 129 people died and 352 people were injured. The attack happened on the evening of 13 November 2015. 6 mass shootings and 3 suicide bombings happened.

What happened and the reactions I’ve seen so far make this the European / French equivalent of September 11 attacks. A lot of people died or were injured, one of the most important cities were affected, politicians say we are in war, on Facebook people show condolence.

Questions we need to ask

The Terrorists: Who were the terrorists? What was their story? How did they become terrorists?
The Weapons: Where did they get the weapons from? Can we prevent that?
Fast reactions: How could the police react that fast?
Future: How do we want to deal with such attacks?

The Terrorists

ISIS claims they were responsible for the attacks. We could stop to think here and many certainly will. It is a terrorist organization and that is what terrorists do. The people were religiously motivated lunatics, so we don’t need to understand them. Right?

Wrong.

People are complicated. They don’t always act consistently. Their environment is critical to form who they are. If you grew up in an environment where you didn’t have reliable access to clean water, food, medical services, education, work, and a perspective on a good live in general, you would certainly be somebody else.

Don’t get me wrong here. I don’t want to talk it little. Everybody has a choice what he can do in his live. What terrorists do is cruel. But just like after September 11, I am pretty sure we (France, Germany, Europe) will spend a lot of time and money in reaction to the attacks. One reaction we will certainly get is demands for more spendings / actions in military and surveillance. Demands for closing borders. What I want to say is that we are trying to intervene in a too late step of the “terror process”. People are not born terrorists. They become terrorists during their lives. We should not prevent terrorists from entering our countries, from blowing up what is important for us. We should prevent people from becoming terrorists.

Now you might say that is easier said than done. But a very first step would be getting a detailed view of the terrorists lives. I don’t mean the preparation, but their complete live. Their environment. And then change that.

My assumption is that we could completely prevent generations of terrorists from being born when we “simply” make the world better.

Weapons

According to the Paris prosecutor, the attackers wearing suicide vests used TATP as an explosive. This is easy to make from things of everyday live. So we can’t prevent it from being made and used by suicide bombers, except if we massively sacrifice privacy. I’m talking here about cameras in your bedroom kind of surveillance. “Only” the US way of surveillance will not change anything.

However, I’ve also read that the terrorists used AK-47 rifles. They cannot be build by yourself, so we should find out where they came from. One only the concrete path, but in general.

Fast Reactions

It is less than two days after the attacks, and we already know which type of explosive the bombing vests were made of, some people were put into arrest, and we are sure that ISIS is responsible. This is astonishing. It feels as if the police was doing an extraordinary good job here. I wonder if it would be that fast outside of Paris.

At the same time, it feels too fast. When we get news that fast, we (unconciously) will expect such fast reactions with other attacks. This might not be possible and people might - too fast - ask for political changes. I just want to point out that such fast reactions and the quick, not very well investigated articles we get today (like this one… but I’m not a journalist) are not always good for us.

Possible actions

I see three ways we, as a society, can react to terrorist attacks in general:

No reaction
Surveillance and military
Development aid

No reaction

I was positively surprised how the people from Norway reacted to the 2011 Norway attacks. They didn’t start new surveillance programs. They didn’t increase their military spendings.

The Norwegians showed they are strong. They showed the world that they are united. They showed that they will not let terrorist attacks change the way they live their lives.

Surveillance and military

When something happens, people want politicians to show they improve the situation. They want revenge. They want others to share their pain. They feel insecure and want to feel strong and secure again.

This is a reason why spending more money in surveillance and military is a very understandable, logical reaction.

But be clear what you’re doing when you ask for more spendings in military and surveillance. Be clear what you sacrifice when you want to expand the power of intelligence services. Think about how you let terrorists change your live. Which other dangers you might put your society into.

I am a strong opponent of surveillance and also (not that much, but still) an opponent of military. One reason is that we are playing a stupid game (see A Waste of Money and Time, a security expert). We see what terrorists did, like an attack with airplanes, and we as a society spend lots and lots of effort to prevent that. We accept being treated like criminals, hundred thousands of passengers, just to get a slight chance to catch a terrorist. The terrorists know that, so they can take the train and do exactly the same. We pick our defenses and they do something different.

I don’t want to play such a stupid game.

One way around is intelligence. But the bad thing is, that it doesn’t make sense to use a little bit of intelligence. You have to go the full, Orwellian way to have a good chance of it being effective. Here is what I could think of:

Make money flows transparent: Don’t use cash any more. Only digital money which gets tracked by government agencies. Track who buys what in which quantities. Make shops, no matter how small, connect the transactions with goods you bought. This has to include everything, as you can use quite a lot of different things to build bombs.
Track everybody: With smartphones you can make detailed movement profiles. Even if you’re not on Facebook, you can build a social graph. Who talked to whom, who has how much contact to whom. Who is influential, who isn’t.
Get rid of privacy: We have surveillance equipment everywhere. Private smartphones, tablets, laptops usually have cameras and microphones. Most people use these devices to communicate. These devices, combined with traffic cameras, private shop / bank cameras give pretty detailed images of what is going on. An automatic computer system like Microsoft’s Domain Awareness System could constantly take this information and react within seconds.

But we have to be aware of what we sacrifice when we are doing it. The surveillance systems can be hacked. They can be misused. The automatic algorithms can - just like humans - make mistakes.

Imagine the world we would live in if the Gestapo or the Stasi had access to this kind of technology. Imagine how people would act differently when they know everything they did was recorded and could be hold against them later.

On the positive side: This could decrease “normal” crime, too.

Development aid

Just as I began to write before, I would prefer if we spend more money on development aid. I am pretty sure there are many (potential) terrorists, which would not even consider that if they were in a better situation. We could massively improve the situation of the live of millions of people. Even if it would not work against terrorism, we would still do something inherently good. We would help the people. If we honestly want to improve their situation, if we don’t use military but humanitarian aid, people will recognize this. People will have something good in their minds when they thought of us (see “Brot statt Bomben”). A positive side-effect would be that it would stop coming more refugees to Europe.

Of course, just throwing money at the problem will not solve anything. We need to check if the money actually gets to the people. We need to check if it improves their lives. We need them to get in a situation where they are able to choose what to do with their lives. Not from equally bad situations, but giving people the possibility to make their live become good.

We need to understand the problems of those countries. We have to work with the people there, not work for them. We have to support, not to lead.

There are certainly a lot of organizations like Engineers Without Borders and Doctors Without Borders which have a very good idea what has to be done in a short-term perspective to improve the lives of many people. However, we should plan for a long-term perspective. We should get in contact with refugees who are willing to get back if the situation was better and try to figure out what they need to improve the situation.

And we should finally stand to our word and reach the Millennium Development Goals I can’t find the source, but the costs for reaching the MDGs were lower than I expected. Especially access to clean water significantly lower than 20 Billion US-Dollar, if I remember it correctly. The GDP of Europe is about 18,527,116 Million US-Dollar. That means we would have to spend about 0.1 % of our GDP for this goal. Now think of the money that was spend in the European debt crisis. It is hard to put numbers on here, but it was much more than 18 Billion US-Dollar.

Hypocracy

A fellow student recently pointed out that there were also 2015 Beirut bombings. I didn’t even see that in the news. A lot of people die preventable deaths due to the lack of access to clean water, sanitation and food. And we don’t even talk about the MDGs. Not really.

Some final words

One main point of this mini-article is that I am pretty sure people will react to fast. No matter what we do, we should think about it thoroughly. This article is - like hundreds of blog articles, quite a bit of “professional” journalist articles and surely some statements by politicians - not well investigated. It is only a quick, spontaneous reaction.

The other important point is that we have alternatives to more military and surveillance spendings. We should really think about that. We should talk about it and evaluate it. Finally, it the question is: In which kind of society do we want to live?

Edit: The reaction

Now it seems to be clear how most politicians react:

Tensor Flow - A quick impression

info@martin-thoma.de (Martin Thoma) — Wed, 11 Nov 2015 22:33:00 +0100

Tensor Flow is a machine learning toolkit which recently got published by Google. They published it under Apache License 2.0. Looking at the source code overview, it seems to be mainly C++ with a significant bit of Python.

I guess the abstract of the Whitepaper is a good description what TensorFlow is:

TensorFlow is an interface for expressing machine learning algorithms, and an implementation for executing such algorithms. A computation expressed using TensorFlow can be executed with little or no change on a wide variety of heterogeneous systems, ranging from mobile devices such as phones and tablets up to large-scale distributed systems of hundreds of machines and thousands of computational devices such as GPU cards. The system is flexible and can be used to express a wide variety of algorithms, including training and inference algorithms for deep neural network models, and it has been used for conducting research and for deploying machine learning systems into production across more than a dozen areas of computer science and other fields, including speech recognition, computer vision, robotics, information retrieval, natural language processing, geographic information extraction, and computational drug discovery. This paper describes the TensorFlow interface and an implementation of that interface that we have built at Google. The TensorFlow API and a reference implementation were released as an open-source package under the Apache 2.0 license in November, 2015 and are available at www.tensorflow.org.

The core seems to be written in C++, but it has a Python front end.

By now, I couldn’t test much because I just made my GPU machine unusable (while trying to get the GPU General Computing practical software to run…). I’ll try to expand this article as soon as possible, but I guess it might take several weeks until I have enough time. Lets see…

Installation

The documentation about the installation makes a VERY good impression. Better than anything I can write in a few minutes, so … RTFM 😜

For Linux systems with CUDA and without root privileges, you can install it with:

$ pip install https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow-0.5.0-cp27-none-linux_x86_64.whl --user

But remember you have to set the environment variable LD_LIBRARY_PATH and CUDA_HOME. For many configurations, adding the following lines to your .bashrc will work:

export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/cuda/lib64"
export CUDA_HOME=/usr/local/cuda

MNIST

The following code can be used to check if your Tensor Flow installation is working. You have to have the get_mnist_data_tf.py in the same directory as the following script. I’ve - more or less - directly copied it from the tutorial. Just execute the script below and see if it finishes without throwing errors.

#!/usr/bin/env python
from get_mnist_data_tf import read_data_sets
mnist = read_data_sets("MNIST_data/", one_hot=True)
import tensorflow as tf
sess = tf.InteractiveSession()
x = tf.placeholder("float", shape=[None, 784])
y_ = tf.placeholder("float", shape=[None, 10])
W = tf.Variable(tf.zeros([784,10]))
b = tf.Variable(tf.zeros([10]))
sess.run(tf.initialize_all_variables())
y = tf.nn.softmax(tf.matmul(x,W) + b)
cross_entropy = -tf.reduce_sum(y_*tf.log(y))
train_step = tf.train.GradientDescentOptimizer(0.01).minimize(cross_entropy)
for i in range(1000):
  batch = mnist.train.next_batch(50)
  train_step.run(feed_dict={x: batch[0], y_: batch[1]})
correct_prediction = tf.equal(tf.argmax(y,1), tf.argmax(y_,1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))

print(accuracy.eval(feed_dict={x: mnist.test.images,
                               y_: mnist.test.labels}))


def weight_variable(shape):
  initial = tf.truncated_normal(shape, stddev=0.1)
  return tf.Variable(initial)

def bias_variable(shape):
  initial = tf.constant(0.1, shape=shape)
  return tf.Variable(initial)

def conv2d(x, W):
  return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')

def max_pool_2x2(x):
  return tf.nn.max_pool(x, ksize=[1, 2, 2, 1],
                        strides=[1, 2, 2, 1], padding='SAME')

W_conv1 = weight_variable([5, 5, 1, 32])
b_conv1 = bias_variable([32])

x_image = tf.reshape(x, [-1,28,28,1])

h_conv1 = tf.nn.relu(conv2d(x_image, W_conv1) + b_conv1)
h_pool1 = max_pool_2x2(h_conv1)

W_conv2 = weight_variable([5, 5, 32, 64])
b_conv2 = bias_variable([64])

h_conv2 = tf.nn.relu(conv2d(h_pool1, W_conv2) + b_conv2)
h_pool2 = max_pool_2x2(h_conv2)

W_fc1 = weight_variable([7 * 7 * 64, 1024])
b_fc1 = bias_variable([1024])

h_pool2_flat = tf.reshape(h_pool2, [-1, 7*7*64])
h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat, W_fc1) + b_fc1)

keep_prob = tf.placeholder("float")
h_fc1_drop = tf.nn.dropout(h_fc1, keep_prob)

W_fc2 = weight_variable([1024, 10])
b_fc2 = bias_variable([10])

y_conv = tf.nn.softmax(tf.matmul(h_fc1_drop, W_fc2) + b_fc2)

cross_entropy = -tf.reduce_sum(y_*tf.log(y_conv))
train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)
correct_prediction = tf.equal(tf.argmax(y_conv,1), tf.argmax(y_,1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))
sess.run(tf.initialize_all_variables())
for i in range(1000):
  batch = mnist.train.next_batch(50)
  if i%100 == 0:
    train_accuracy = accuracy.eval(feed_dict={
        x:batch[0], y_: batch[1], keep_prob: 1.0})
    print("step %d, training accuracy %g"%(i, train_accuracy))
  train_step.run(feed_dict={x: batch[0], y_: batch[1], keep_prob: 0.5})

print("test accuracy %g"%accuracy.eval(feed_dict={
    x: mnist.test.images, y_: mnist.test.labels, keep_prob: 1.0}))

Observations

While looking at the MNIST example, I made a couple of observations. Let’s begin with the nice parts:

Tensor Flow has a usable documentation (e.g. The neural network part). Not great, as Lasagne where you have lots of details (e.g. activation functions)
Seems to be quite easy to use.
Seems to be well-tested by simply being used in many different projects by Google.
Just like Theano (and thus Lasagne), Tensor flow has automatic differenciation.

Not sure:

How easy is it to share trained models? In which format would you do so?
How easy is it to understand a shared model?
How easy is it to get something new to Tensor Flow like recurrent layers? (Actually, this seems rather to show that either the Whitepaper is a bit misleading or the documentation / Google search is not that good. In the whitepaper they write something about LTSM models, but I couldn’t find any docs about that. Only by manually going through the manual, I found it)

Not so nice:

Doesn’t work with Python 3.
They don’t follow PEP8. I know that there is a Python style guide by Google, but it does not seem to follow that one either. See the next section for some more detailed feedback.
Just like the other Toolkits, you need CUDA. It doesn’t work with OpenCL.

PEP8

Whitespace
- W = tf.Variable(tf.zeros([784,10])) should be W = tf.Variable(tf.zeros([784, 10])). Missing whitespaces happened quite often.
- Indent with 2 spaces instead of 4 spaces. The Google guide seems also to use 4 (example).
- Newlines between functions are missing.
Print statement instead of a print function was used &rightarrow; only Python 2, not Python 3.
I’m not sure why y_ has the trailing underscore. According to PEP8, a single trailing underscore is used by convention to avoid conflicts with Python keyword.
A mixture of different styles as pointed out on Credric’s Blog

Videos

Starting at 21m 2s:

Alternatives / Similar software

As I don’t really know by now what Tensor Flow is doing, I can’t pin-point alternatives. But I have some educated guesses:

Theano has been around for quite a while and seems to have a similar approach with its computational graph. Enhanced by Lasagne, it is a pretty good alternative when it comes to neural networks. Lasagne has an exceptionally good documentation, but parts of the tutorial could still be improved.
Caffe was something I recently tried. I didn’t like it too much due to the lack of documentation, but it certainly is a big project. Especially when it comes to images.
I haven’t tried, but they look promising:
- Chainer
- MXNet
- CGT
- Torch has a very nice example for a character predictor.

Machine Learning 1

info@martin-thoma.de (Martin Thoma) — Mon, 09 Nov 2015 16:02:00 +0100

Dieser Artikel beschäftigt sich mit der Vorlesung „Machine Learning 1“ am KIT. Er dient als Prüfungsvorbereitung. Ich habe die Vorlesungen bei Herrn Prof. Dr. Zöllner im Wintersemester 2014/2015 gehört.
Es gibt auch einen Artikel über Machine Learning 2

Folien

Einordnungskriterien

Slide name: ML-Einordnungskriterien.pdf

Algorithmus		Inferenztyp		Lernebene		Lernvorgang		Beispielgebung		Beispielumfang		Hintergrundwissen
Algorithmus		ind.	ded.	symb.	subsymb.	überwacht	unüb.	inkr.	nicht inkr.	gering	groß	emp.	axio.
$k$-NN		x			x	x			x		x	x
SVMs		x			x	x			x		x	x
Decision Trees	ID3	x		x		x			x		x	x
Decision Trees	ID5R	x		x		x		x			x	x
NN	klassisch	x			x	x			x		x	x
NN	Auto-Encoder	x			x		x		x		x	x
Bayessche Netze		x			x	x			x		x	x
HMMs		x			x	x			x		x	x
Version-Space Algorithmus		x		x		x		x			x	x
Specific-to-General Konzeptlernen		?	?	x		?	?	?	?	?	?	?	?
k-means clustering		x			x		x		x		x	x
AHC		x			x		x		x		x	x
COBWEB		x		x			x	x		x		x
CBR		x		x		x		x		x		x
EBG			x	x		x		x		x			x

Einführung

Slide name: MLI_01_Einfuehrung_slides1.pdf

Was ist Intelligenz? (Problemlösen, Erinnern, Sprache, Kreativität, Bewusstsein, Überleben in komplexen Welten, )
Wissensrepräsentation:
- Assoziierte Paare (Eingangs- und Ausgangsvariablen)
- Entscheidungsbäume (Klassen diskriminieren)
- Parameter in algebraischen ausdrücken
- Formale Grammatiken
- Logikbasierte Ausdrücke
- Taxonomien
- Semantische Netze
- Markov-Ketten

Machine Learning von Tom Mitchell: A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.
Deduktion: Die Deduktion ist eine Schlussfolgerung von gegebenen Prämissen auf die logisch zwingenden Konsequenzen. Deduktion ist schon bei Aristoteles als „Schluss vom Allgemeinen auf das Besondere“ verstanden worden.
Modus ponens: Der Modus ponens ist eine Art des logischen Schließens. Er besagt: Wenn die Prämissen $A \rightarrow B$ und $A$ gelten, dann gilt auch $B$.
Abduktion by Peirce: Deduction proves that something must be; Induction shows that something actually is operative; Abduction merely suggests that something may be.

Induktives Lernen

Slide name: MLI_02_InduktivesLernen_slides1.pdf

Version Space: Der Raum aller Hypotesen, welche mit den Trainingsbeispielen konsistent sind.
Version Space Algorithmus: Der Version Space Algorithmus ist ein binärer Klassifikator für diskrete Feature-Spaces. Er startet mit der generellsten Hypothese $G = (?, \dots, ?)$ - alles ist wahr - und der speziellsten Hypothese $S = (\#, \dots, \#)$ - nichts ist wahr. Wenn ein Beispiel mit dem Label true gesehen wird, dann wird die speziellste Hypothese angepasst und veralgemeinert. Wenn ein Beispiel mit dem Label false gesehen wird, wird die generellste Hypothese spezialisiert.
So kann man den Raum aller mit den Trainingsdaten konsistenten Hypothesen finden.
Konzept: Ein Konzept beschreibt eine Untermenge von Objekten oder Ereignissen definiert auf einer größerer Menge.
Konsistenz: Keine negativen Beispiele werden positiv klassifiziert.
Vollständigkeit: Alle positiven Beispiele werden als positiv klassifiziert.

Algorithmen: Bäume (Wälder?)
- Suche vom Allgemeinen zum Speziellen: Negative Beispiele führen zur Spezialisierung
- Suche vom Speziellen zum Allgemeinen: Positive Beispiele führen zur Verallgemeinerung
- Version Space: Beides gleichzeitig anwenden
Präzendenzgraphen: In welcher Reihenfolge werden Aktionen ausgeführt?

Version Space Algorithmus ist:

Induktiver Inferenztyp
Symbolische Ebene des Lernens
Überwachtes Lernen
Inkrementelle Beispielgebung
Umfangreich (viele Beispiele)
Empirisches Hintergrundwissen
Voraussetzungen: Konsistente Beispiele, korrekte Hypothese im Hypothesenraum
Positive Aspekte:
- Es ist feststellbar, welche Art von Beispielen noch nötig ist
- Es ist feststellbar, wann das Lernen abgeschlossen ist

Weiteres

Inductive bias: Induktives Lernen benötigt Vorannahmen.
Bias ("Vorzugskriterium"): Vorschrift, nach der Hypothese gebildet werden.

Reinforcement Learning

Slide name: MLI_03_ReinforcementLearning_slides1.pdf

Siehe auch:

Neuronale Netze
Machine Learning 2
Cat vs. Mouse code
Berkeley
- CS188 Intro to AI: Project 3: Reinforcement Learning
- Dan Klein, Pieter Abbeel: Lecture 10: Reinforcement Learning on YouTube. University of California, Berkeley. This expalins TD-learning.
What is the Q function and what is the V function in reinforcement learning?

Markovsches Entscheidungsproblem (Markov Decision Process, MDP)

Ein Markovsches Entscheidungsproblem ist ein 5-Tupel $S, A, P, R, \gamma$ mit:

$S$: Endliche Zustandsmenge (states)
$A(s)$: Die Menge von möglichen Aktionen im Zustand $s$
$P(s, s', a) = P(s_{t+1} = s' | s_t = s, a_t = a)$: Die Wahrscheinlichkeit im Zeitschritt $t+1$ im Zustand $s'$ zu sein, wenn man zum Zeitpunkt $t$ im Zustand $s$ ist und die Aktion $a$ ausführt
$R(s, s', a) \in \mathbb{R}$: Die direkte Belohnung, wenn durch die Aktion $a$ vom Zustand $s$ in den Zustand $s'$ gekommen ist.
$\gamma \in [0, 1]$: Der Diskontierungsfaktor, welche die Bedeutung von direkten Belohnungen im Vergleich zu künftigen Belohnungen anzeigt.

Reinforcement Learning (RL, Bestärkendes Lernen)

Beim bestärkenden Lernen liegt ein Markow-Entscheidungsproblemen vor. Es gibt also einen Agenten, der Aktionen ausführen kann. Diese können (nicht notwendigerweise sofort) bewertet werden.

Policy

Eine policy $\pi: S \rightarrow A$ ist die Vorschrift, in welchem Zustand welche Aktion ausgeführt werden soll.

Policy Learning

Unter Policy Learning versteht man die Suche nach einer optimalen Policy $\pi^*$.

Value-Funktion

Die Funktion $V^\pi: S \rightarrow \mathbb{R}$ heißt Value-Funktion. Sie gibt den erwarteten Wert (nicht die Belohnung, da bei der V-Funktion noch der Diskontierungsfaktor eingeht!) eines Zustands $s$ unter der policy $\pi$ an. Mit $V^*$ wird der Wert unter der optimalen policy bezeichnet.

Q-Funktion

Die Funktion $Q^\pi: S \times A \rightarrow \mathbb{R}$ gibt den erwarteten Wert einer eines Zustandes $s$ unter der policy $\pi$, wenn die Aktion $a$ ausgeführt wird an. Es gilt: \[Q^\pi(s, \pi(s)) = V^\pi(s)\]

Eligibility Traces

Die Idee scheint einfach zu sein, dass man ein späteres Update auch auf frühere Ereignisse "zurückpropagiert". TODO See also: Reinforcement Learning: An Introduction by Sutton.

Beispiel für RL: Roboter muss zu einem Ziel navigieren

Algorithmen:

Simple Value Iteration

Simple Value Iteration estimates the value function by updating it as long as necessary to converge: \[\hat{V}^*(s_t) \leftarrow r_t + \gamma \hat{V}^*(s_{t+1})\] "Simple" means that the transition function is deterministic. It is explained in

Sebastian Thrun: Unit 9 17 Value Iteration 1 on YouTube.
Sebastian Thrun: Unit 9 17 Value Iteration 2 on YouTube.
Sebastian Thrun: Unit 9 17 Value Iteration 3 on YouTube.

Simple Temporal Difference Learning

Simple Temporal Difference Learning is just like Simple Value Iteration, but now the Value function is updated with a learning rate $\alpha$: \[\hat{V}^*(s_t) \leftarrow (1-\alpha) \cdot \hat{V}^*(s_t) + \alpha(r_t + \gamma \hat{V}^*(s_{t+1}))\] Mehr dazu im nächsten Abschnitt.

Q-Learning

Siehe nächster Abschnitt

SARSA (State-Action-Reward-State-Action)

SARSA is a learning algorithm which updates the Q-function: \[Q(s_t,a_t) \leftarrow (1-\alpha) \cdot Q(s_t,a_t) + \alpha [r_{t+1} + \gamma Q(s_{t+1}, a_{t+1})]\] where $\alpha \in (0, 1)$ is the learning rate and $\gamma \in [0, 1]$ is the discount factor.

SARSA($\lambda$)

SARSA($\lambda$) ist SARSA mit Eligibility Traces. TODO

Q-Learning

Q-learning
- YouTube: Lecture 18: RL Part 1: Q-Learning: 1:16:11. BrownCS141 Spring 2014.
- YouTube: PacMan

Pseudocode:

initialize Q[num_states, num_actions]
start in state s
repeat:
    select and execute action a
    r ← R(s, a)  # Receive reward
    s' ← T(s, a) # Get on new state
    Q[s', a] ← (1- α) * Q[s, a] + α * (r + γ max_{a'} Q[s', a'])
    s ← s'

where $\alpha \in (0, 1]$ is a learning rate and $\gamma$ is a discount factor.

See also:

Mario Q-learning on YouTube. 2010.

TD-Learning

R. Sutton und A. Barto: Temporal-Difference Learning. 1998.

Der TD-Learning Algorithmus beschäftigt sich mit dem Schätzen der Value-Funktion $V^\pi$ für eine gegebene Policy $\pi$. Das wird auch policy evaluation oder prediction genannt.

TD-Learning (Temporal Difference Learning)

Siehe auch

Optimalitätsprinzip von Bellman
Guest Post (Part I): Demystifying Deep Reinforcement Learning

Lerntheorie

Slide name: MLI_04_Lerntheorie_slides1.pdf

Ockhams Rasiermesser (Quelle: Wikipedia): Von mehreren möglichen Erklärungen für ein und denselben Sachverhalt ist die einfachste Theorie allen anderen vorzuziehen. Eine Theorie ist einfach, wenn sie möglichst wenige Variablen und Hypothesen enthält, und wenn diese in klaren logischen Beziehungen zueinander stehen, aus denen der zu erklärende Sachverhalt logisch folgt.
Structural Risc Minimization (SRM): Unter Structural risk minimization versteht man die Abwägung zwischen einem einfachen Modell und einem komplexen Modell, welches auf den Trainingsdaten besser funktioniert aber eventuell mehr unter Overfitting leidet.
Vapnik-Chervonenkis Dimension (VC-Dimension): Die VC-Dimension $VC(H, X) \in \mathbb{N} \cup \infty$ eines Hypothesenraumes $H$ ist gleich der maximalen Anzahl an Datenpunkten aus $X$, die von $H$ beliebig in zwei Mengen gespalten werden können. Dabei muss es nur eine Teilmenge $X' \subseteq X $ der Größe $n$ geben, damit $VC(H, X) \geq n$ gilt. Falls beliebige Teilmengen von $X$ durch $H$ separiert werden können, so gilt $VC(H, X) = \infty$. Praktisch gesehen ist $X$, die Menge aller möglichen Features, sowie $H$, die Menge aller möglichen Trennlinien im Feature-Space, vorgegeben. Die Frage ist ob man eine Teilmenge $X' \subseteq X$ findet mit $|X'| = n$, sodass man für $X'$ jede Mögliche Teilung in zwei Mengen durch $H$ realisieren kann.
Probably approximately correct learning (PAC): PAC macht eine Aussage über die Anzahl der benötigten Stichproben, wenn man einen bestimmten realen Fehler mit einer frei zu wählenden Wahrscheinlichkeit bekommen will.

Lernmaschine wird definiert durch Hypothesenraum ${h_\alpha: \alpha \in A}$ und Lernverfahren. Das Lernverfahren ist die Methode um $\alpha_{\text{opt}}$ mit Hilfe von Lernbeispielen zu finden.
Probleme beim Lernen:
- Größe des Hypothesenraums im Vergleich zur Anzahl der Trainingsdaten.
- Das Verfahren könnte nur suboptimale Lösungen finden.
- Das Verfahren könnte die passende Hypothese nicht beinhalten.
Lernproblemtypen: Sei die Menge der Lernbeispiele in $X \times Y$, mit $X \times Y =$…
- $\{Attribut_1, Attribut_2, …\} \times \{True, False\}$: Konzeptlernen
- $\mathbb{R}^n \times \{Klasse_1, …, Klasse_n\}$: Klassifikation
- $\mathbb{R}^n \times \mathbb{R}$: Regression
Gradientenabstieg, Overfitting
Kreuzvalidierung
PAC
- Folie 35: Was ist eine Instanz der Länge $n$?
  Eine Hypothese mit $n$ Literalen.

Boosting

Boosting: Kombiniere mehrere schwache Modelle um ein gutes zu bekommen, indem Trainingsbeispiele unterschiedlich gewichtet werden.
Bagging (Bootstrap aggregating): Kombiniere mehrere schwache Modelle um ein gutes zu bekommen. Dabei bekommt jedes schwache Modell nur eine Teilmenge aller Trainingsdaten.
AdaBoost (Adaptive Boosting; see YouTube): Learn a classifier for data. Get examples where the classifier got it wrong. Train new classifier on the wrong ones.

Folie 22:
- Wofür steht $i$ und welchen Wertebereich hat $i$?
  → $i$ ist eine Zählvariable, welche die Trainingsdaten durchnummeriert.
- Stellt $W_k(i)$ die Wahrscheinlichkeit dar, dass Beispiel $i$ im $k$-ten Durchlauf für das Training verwendet wird?
  → Nein. $W_k(i)$ ist das Gewicht des $i$-ten Trainingsbeispiels für den $k$-ten klassifikator. Siehe Folie 24 und folgende für ein Beispiel.

Siehe auch:

Alexander Ihler: AdaBoost.

Ensemble Learning Techniques: Boosting, Bagging, Random Subspaces, Pasting, Random Patches

VC-Dimension

VC-Dimension, siehe YouTube und [Mit97]: Sei $H^\alpha = \{h_\alpha : \alpha \in A\}$ der Hypothesenraum. Die VC-Dimension $VC(h_\alpha)$ von $H^\alpha$ ist gleich der maximalen Anzahl von beliebig platzierten Datenpunkten, die von $H^\alpha$ separiert werden können.

Folie 44: $\eta \in [0, 1]$ ist ein Parameter, der beliebig gewählt werden kann. Siehe Info-Box Abschätzung des realen Fehlers.

Neuronale Netze

Slide name: MLI_05_Neuronale_Netze_slides1.pdf

Einsatzfelder:
- Klassifiktion: Spracherkennung, Schrifterkennung
- Funktionsapproximation
- Mustervervollständigung: Kodierung, Bilderkennung (NODO: Warum zählt das nicht zu Klassifikation?)
Perzeptron von Rosenblatt (1960)
- Auswertung: Input-Vektor und Bias mit Gewichten multiplizieren, addieren und Aktivierungsfunktion anwenden.
- Training: Zufällige Initialisierung des Gewichtsvektors, addieren von fehlklassifizierten Vektoren auf Gewichtsvektor.
Gradientenabstieg
Software:
- Lasagne: Python, hat eine exzellente Dokumentation, die auch größtenteils auf explizit auf Literatur verweist und die Formeln hinter den Funktionen direkt angibt.
- Google TensorFlow

Cascade Correlation

Cascade Correlation ist ein konstruktiver Algorithmus zum erzeugen von Feed-Forward Neuronalen Netzen. Diese haben eine andere Architektur als typische multilayer Perceptrons. Bei Netzen, welche durch Cascade Correlation aufgebaut werden, ist jede Hidden Unit mit den Input-Neuronen verbunden, mit den Output-Neuronen und mit allen Hidden Units in der Schicht zuvor.

RPROP (siehe YouTube - 15:00min)

Rprop ist eine Gewichtsupdate-Regel für neuronale Netze. Sie betrachtet nur das Vorzeichen des Gradienten, jedoch nicht den Betrag. Jedes Gewicht wird unabhängig von den anderen behandelt. Der Algorithmus hat Konstanten $\eta^- \in \mathbb{R}_{\le 1}$ sowie $\eta^+ \in \mathbb{R}_{\ge 1}$. Für jedes Gewicht ist außerdem $\eta=1$ zu Beginn. Bei jedem Gewichtsupdate wird überprüft, ob sich das Vorzeichen des Gradienten für dieses Gewicht geändert hat. Falls ja, wird das Gewicht um $\eta \cdot \eta^+$ bzw $\eta \cdot \eta^-$ geändert. Außerdem kann eine minimale bzw. eine Maximale Änderung gesetzt werden.

Delta-Regel, siehe neuronalesnetz.de

Die Delta-Regel ist ein Lernalgorithmus für neuronale Netze mit nur einer Schicht. Sie ist ein Spezialfall des algemeineren Backpropagation-Algorithmus und lautet wie folgt: \[\Delta w_{ji} = \alpha (t_j - y_j) \varphi'(h_j) x_i\] wobei

$\Delta w_{ji} \in \mathbb{R}$ die Änderung des Gewichts von Input $i$ zum Neuron $j$,
$\alpha \in [0, 1]$ die Lernrate (typischerweise $\alpha \approx 0.1$),
$t_j \in \mathbb{R}$ der Zielwert des Neurons $j$,
$y_j \in \mathbb{R}$ die tatsächliche Ausgabe,
$\varphi'$ die Ableitung der Aktivierungsfunktion des Neurons,
$h_j \in \mathbb{R}$ die gewichtete Summe der Eingaben des Neurons und
$x_i \in \mathbb{R}$ der $i$-te Input

ist.

Gradient-Descent Algorithmus

Der Gradient-Descent Algorithmus ist ein Optimierungsalgorithmus für differenzierbare Funktionen. Er startet an einer zufälligen Stelle $x_0$. Dann wird folgender Schritt mehrfach ausgeführt: \[x_0 \gets x_0 - \alpha \cdot \text(grad) f (x_0)\] wobei $\alpha \in (0, 1]$ die Lernrate ist und $f$ die zu optimierende Funktion. Dabei könnte $\alpha$ mit der Zeit auch kleiner gemacht werden.

Backpropagation (siehe neuralnetworksanddeeplearning.com)

Der Backpropagation-Algorithmus ist eine Variante des Gradient-Descent Algorithmus, welche für MLPs angepasst wurde. Sie besteht aus drei Schritten:

Forward-Pass: Lege die Input-Features an das Netz an und erhalte den Output
Fehlerberechnung: Mache das für alle Daten
Backward-Pass: Passe die Gewichte

Im Grunde ist Backpropagation nur eine Geschwindigkeitsoptimierte Variante des Gradient-Descent Algorithmus, da die Gradienten im Backpropagation-Algorithmus auf geschickte Weise berechnet werden.

Radiale Basisfunktion (Radial Basis Function, RBF)

Eine radiale Basisfunktion ist eine Funktion $f: D \rightarrow \mathbb{R}$, für die $f(x) = f(\|x\|)$ gilt bzw. allgemeiner, für die ein $c \in D$ existiert, sodass $f(x, c) = f(\|x - c\|)$ gilt. Der Wert der Funktion hängt also nur von der Distanz zum Ursprung bzw. allgemeiner zu einem Punkt $c \in D$ ab. Ein typisches Beispiel sind gaußsche RBFs: $f(x) = e^{-(a (x - c)^2)}$, wobei $a, c$ Konstanten sind.

Radial-Basis Funktion Netz (RBF-Netz)

Ein Radial-Basis Funktion Netz ist eine neuronales Netz, welches als Aktivierungsfunktionen RBFs verwendet. Dabei gibt es dann für jedes Neuron im Grunde zwei Parameter: Der Radius und das Zentrum (vgl. Folie 39 für die Gewichtsanpassung).

Dynamic Decay Adjustment (DDA)

DDA ist ein konstruktiver Lernalgorithmus für RBF-Netze welcher in [Ber95] vorgestellt wird. Bei den Netzwerken, die DDA annimmt, gibt es sog. Prototypen. Das scheinen einfach Neuronen mit RBF-Aktivierungsfunktionen zu sein, welche für eine Klasse stehen. Zwei Schwellwerte, $\theta^+$ und $\theta^-$, werden eingeführt. Der Schwellwert $\theta^+$ muss beim Training eines Beispiels der Klasse $y_1$ von einem Neuron der Klasse $y_1$ überschritten werden. Falls das nicht der Fall ist, wird ein neues Neuron hinzugefügt.
Der Schwellwert $\theta^-$ ist eine obere Grenze für die Aktivierung von Neuronen, die zu anderen Klassen gehören. Ist eine Aktivierung höher, wird der Radius des zugehörigen Neurons verringert.
$\theta^+ = 0.4$ und $\theta^- = 0.2$ sind sinnvolle Werte.
Laut einem Prüfungsprotokoll lernt DDA nach Vapnik korrekt.

Siehe auch: The Dynamic Decay Adjustment Algorithm

Siehe auch

Instanzbasiertes Lernen

Slide name: MLI_06_InstanzbasiertesLernen_slides1.pdf

Instanzenbasiertes Lernen bzw. Lazy Learning: Instanzenbasiertes Lernen ist ein Lernverfahren, welches einfach nur die Beispiele abspeichert, also faul (engl. lazy) ist. Soll der Lerner neue Daten klassifizieren, so wird die Klasse des ähnlichsten Datensatzes gewählt.
Case-based Reasoning bzw. kurz CBR: CBR ist ein allgemeines, abstraktes Framework und kein direkt anwendbarer Algorithmus. Die Idee ist, dass nach ähnlichen, bekannten Fällen gesucht wird, auf die der aktuelle Fall übertragen werden kann.
Fall im Kontext des CBR: Ein Fall ist eine Abstraktion eines Ereignisses, die in Zeit und Raum begrenzt ist. Ein Fall enthält eine Problembeschreibung, eine Lösung und ein Ergebnis. Zusätzlich kann ein Fall eine Erklärung enthalten warum das Ergebnis auftrat, Informationen über die Lösungsmethode, Verweise auf andere Fälle oder Güteinformationen enthalten.

Beispiel für Lazy Learning: (k)-NN, CBR
NODO: Folie 3: „Fleißige“ Lernalgorithmen mit dem gleichen Hypothesenraum sind eingeschränkter - was ist damit gemeint? Was sind fleißige Lernalgorithmen? Lernalgorithmen, welche den meisten Rechenaufwand beim Lernen investieren, wo aber das auswerten vergleichsweise billig ist?

SVM

Slide name: MLI_07_SVM_slides1.pdf

Eine Erklärung von SVMs findet sich im Artikel Using SVMs with sklearn.

SVMs sind laut Vapnik die Lernmaschine mit der kleinsten möglichen VC- Dimension, falls die Klassen linear trennbar sind.
Primäres Optimierungsproblem: Finde einen Sattelpunkt der Funktion
$L_P = L(\vec{w}, b, \vec{\alpha}) = \frac{1}{2}|\vec{w}|^2 - \sum_{i=1}^N \alpha_i (y_i(\vec{w}\vec{x_i}+b)-1)$ wobei $\alpha_1, \dots, \alpha_N \geq 0$ Lagrange-Multiplikatoren sind
Soft Margin Hyperebene
Der Parameter (C) dient der Regularisierung. Ist (C) groß gibt es wenige Missklassifikationen in der Trainingsdatenmenge. Ist (C) klein, werden die Margins größer.
Nichtlineare Kernelmethoden
Kernel-Trick

Abschätzung des realen Fehlers

Der reale Fehler kann durch den empirischen Fehler und die VC-Dimension wie folgt abgeschätzt werden: Mit Wahrscheinlichkeit $P(1-\eta)$ gilt: \[E(h_\alpha) \leq E_{emp}(h_\alpha) + \sqrt{\frac{VC(h_\alpha)}{N} \cdot (\log(2 N / VC(h_\alpha)) + 1) - \frac{\log(\eta / 4)}{N}}\] wobei gilt:

$E(h_\alpha)$ ist der reale Fehler der mit der Hypothese $h_\alpha$ gemacht wird
$E_{emp}(h_\alpha)$ ist der empirische Fehler der mit der Hypothese $h_\alpha$ gemacht wird
$VC(h_\alpha)$ ist die VC-Dimension der Lernmaschine
$N$ ist die Anzahl der Lernbeispiele
$0 \leq \eta \leq 1$

Dieser Term wird in der Structural Risc Minimization minimiert.

Entscheidungsbäume

Slide name: MLI_08_Entscheidungsbaeume_slides1.pdf

Entscheidungsbaum (Decision Tree): Ein Entscheidungsbaum ist ein Klassifikator in Baumstruktur. Die inneren Knoten des Entscheidungsbaumes sind Attributtests, die Blätter sind Klassen.

Typischerweise wird ein Entscheidungsbaum aufgebaut, indem das jeweilige Attribut mit dem höchsten Information Gain als nächster Knoten hinzugefügt wird. Siehe Information gain in decision trees für weitere Informationen.
ID3 (siehe pseudocode): ID3 ist ein Top-Bottom Verfahren zum Aufbau eines Entscheidungsbaumes.
C4.5 (siehe pseudocode): ID3 ist ein Top-Bottom Verfahren zum Aufbau eines Entscheidungsbaumes, welches auf ID3 basiert.
Random Forest, Quelle: Wikipedia: Ein Random Forest ist ein Klassifikationsverfahren, welches aus mehreren verschiedenen, unkorrelierten Entscheidungsbäumen besteht. Alle Entscheidungsbäume sind unter einer bestimmten Art von Randomisierung während des Lernprozesses gewachsen. Für eine Klassifikation darf jeder Baum in diesem Wald eine Entscheidung treffen und die Klasse mit den meisten Stimmen entscheidet die endgültige Klassifikation.

Der Algorithmus ID5R dienen dem Aufbau eines Entscheidungsbaumes.
C4.5 unterstützt - im Gegensatz zu ID3 - kontinuierliche Attributwerte. Außerdem kann C4.5 mit fehlenden Attributwerten umgehen.
Mögliches Qualtitätsmaß ist Entropie:
$Entropie(S) = - p_\oplus \log_2 p_\oplus - p_\ominus \log_2 p_\ominus$ wobei $\oplus$ die positiven Beispiele und $\ominus$ die negativen Beispiele bezeichnet.
Folie 41: Wo ist der Vorteil von ID5R im Vergleich zu ID3, wenn das Ergebnis äquivalent ist?
→ ID5R kann inkrementell verwendet werden. Es ist bei ID5R - im Gegensatz zu ID3 - also nicht nötig bei neuen Trainingsdaten neu zu trainieren.
Random Forest: Erstelle mehrere Entscheidungsbäume mit einer zufälligen Wahl an Attributen. Jeder Baum stimmt für eine Klasse und die Klasse, für die die meisten Stimmen, wird gewählt.

Bayes Lernen

Slide name: MLI_09_BayesLernen_slides1.pdf

Siehe auch:

Dynamische Bayesssche Netze in ML2

Satz von Bayes

Seien $A, B$ Ereignisse, $P(B) > 0$. Dann gilt: $P(A|B) = \frac{P(B|A) \cdot P(A)}{P(B)}$
Dabei wird $P(A)$ a priori Wahrscheinlichkeit, $P(B|A)$ likelihood, und $P(A|B)$ a posteriori Wahrscheinlichkeit genannt.

Naiver Bayes-Klassifikator

Ein Klassifizierer heißt naiver Bayes-Klassifikator, wenn er den Satz von Bayes unter der naiven Annahme der Unabhängigkeit der Features benutzt.

Produktregel

$P(A \land B) = P(A|B) \cdot P(B) = P(B|A) \cdot P(A)$

Summenregel

$P(A \lor B) = P(A) + P(B) - P(A \land P)$

Theorem der totalen Wahrscheinlichkeit

Es seien $A_1, \dots, A_n$ Ereignisse mit $i \neq j \Rightarrow A_i \cap A_j = \emptyset \;\;\;\forall i, j \in 1, \dots, n$ und $\sum_{i=1}^n A_i = 1$. Dann gilt:
$P(B) = \sum_{i=1}^n P(B|A_i) P(A_i)$

Maximum A Posteriori Hypothese (MAP-Hypothese)

Sei $H$ der Raum aller Hypothesen und $D$ die Menge der beobachteten Daten. Dann heißt
$h_{MAP} = \text{arg max}_{h \in H} P(h|D) \cdot P(h)$
die Menge der Maximum A Posteriori Hypothesen.

Maximum Likelihood Hypothese (ML-Hypothese)

Sei $H$ der Raum aller Hypothesen und $D$ die Menge der beobachteten Daten. Dann heißt
$h_{ML} = \text{arg max}_{h \in H} P(h|D)$
die Menge der Maximum Likelihood Hypothesen.

Normalverteilung

Eine stetige Zufallsvariable $X$ mit der Wahrscheinlichkeitsdichte $f\colon\mathbb{R}\to\mathbb{R}$, gegeben durch
$f(x) = \frac {1}{\sigma\sqrt{2\pi}} e^{-\frac {1}{2} \left(\frac{x-\mu}{\sigma}\right)^2}$
heißt $\mathcal N\left(\mu, \sigma^2\right)$-verteilt, normalverteilt mit den Erwartungswert $\mu$ und Varianz $\sigma^2$.

Prinzip der minimalen Beschreibungslänge

Das Prinzip der minimalen Beschreibungslänge ist eine formale Beschreibung von Ockhams Rasiermesser. Nach diesem Prinzip werden Hypothesen bevorzugt, die zur besten Kompression gegebener Daten führen.

Gibbs-Algorithmus (stats.stackexchange)

Der Algorithmus von Gibbs ist eine Methode um Stichproben von bedingten Verteilungen zu erzeugen.

Bedingte Unabhängigkeit

Seien $X, Y, Z$ Zufallsvariablen. Dann heißt $X$ bedingt unabhängig von $Y$ gegeben $Z$, wenn \[P(X|Y,Z) = P(X|Z)\] gilt.

Add $k$ smoothing

Unter Add-$k$-smoothing versteht man eine Technik, durch die sichergestellt wird, dass die geschätzte Wahrscheinlichkeit für kein Ereignis gleich null ist. Wenn man $d \in \mathbb{N}$ mögliche Ergebnisse eines Experiments hat, $N \in \mathbb{N}$ experimente durchgeführt werden, dann schätzt man die Wahrscheinlichkeit von dem Ergebnis $i$ mit \[\hat{\theta_i} = \frac{x_i + k}{N+ kd}, \] wobei $x_i$ die Anzahl der Beobachtungen von $i$ ist und $k \geq 0$ der Glättungsparameter ist.

Bayessches Netz (Quelle: [Dar09])

Ein bayessches Netz ist ein Tupel $(G, \Theta)$ mit:

$G = (\mathbf{X}, E)$ ist ein DAG der Struktur des Bayesschen Netzwerks genant wird. Dabei ist $\mathbf{X} = \{X_1, X_2, \dots, X_n\}$ die Menge der Knoten. Jeder Knoten entspricht einer Zufallsvariablen. Existiert eine gerichtete Kante $(X_i, X_j) \in E$, so existiert eine direkte Abhängigkeit zwischen $X_i$ und $X_j$.
$Theta$ ist die Menge der bedingten Wahrscheinlichkeitsverteilungen und heißt Parametrisierung des bayesschen Netzwerks. Es existiert für jedes $X_i$ genau eine Verteilung in $\Theta$, welche in Abhängigkeit der Elternknoten beschrieben wird.

In einem bayesschem Netz berechnet sich die gemeinsame Verteilung wie folgt: \[P(X_1, \dots, X_N) = \prod_{i=1}^N P(X_i | \text{Eltern}(X_i))\] Die Modelierung von Bayesschen Netzen erfolgt meist durch den Menschen mit Expertenwissen. Alternativ kann die Struktur durch MCMC bestimmt werden. Sobald die Struktur gegeben ist wird die Menge der Verteilungen $\Theta$ durch den Expectation Maximization Algorithmus bestimmt.

Fragen:

Folie 23: Warum ist $h_{MAP(x)}$ nicht die wahrscheinlichste Klassifikation?
Folie 24: Was ist $V$?
Is there any domain where Bayesian Networks outperform neural networks?

HMM

Slide name: MLI_10_HMM_slides1.pdf

Markov-Bedingung (Beschränkter Horizont)

$P(q_{t+1}=S_{t+1}|q_t = S_t, q_{t-1} = S_{t-1}, \dots) = P(q_{t+1}=S_{t+1}|q_t = S_t)$

Hidden Markov Modell (HMM)

Eine HMM ist ein Tupel $\lambda = (S, V, A, B, \Pi)$:

$S = \{S_1, \dots, S_n\}$: Menge der Zustände
$V = \{v_1, \dots, v_m\}$: Menge der Ausgabezeichen
$A \in [0,1]^{n \times n}$ = (a_{ij}): Übergangsmatrix, die die Wahrscheinlichkeit von Zustand $i$ in Zustand $j$ zu kommen beinhaltet
$B = (b_{ik})$ die Emissionswahrscheinlichkeit $v_k$ im Zustand $S_i$ zu beobachten
$\Pi = (\pi_i) = P(q_1 = i)$: Die Startverteilung, wobei $q_t$ den Zustand zum Zeitpunkt $t$ bezeichnet

Vorwärts-Algorithmus

Der Vorwärts-Algorithmus löst das Evaluierungsproblem. Er benutzt dazu dynamische Programmierung: Die Variablen $\alpha_t(i) = P(o_1 o_2 \dots o_t; q_t = s_i | \lambda)$ gibt die Wahrscheinlichkeit an zum Zeitpunkt $t \in 1 \leq t \leq T$ im Zustand $s_i \in S$ zu sein und die Sequenz $o_1 o_2 \dots o_t$ beobachtet zu haben. Diese werden rekursiv berechnet. Dabei beginnt man mit Zeitpunkt $t=1$, berechnet die Wahrscheinlichkeit $o_1$ beobachtet zu haben für jeden Zustand.
Die Wahrscheinlichkeit der beobachteten Sequenz, gegeben die HMM $\lambda$, ist dann einfach die Summe der $\alpha_i$ des letzten Zeitschritts.

Rückwärts-Algorithmus

Der Rückwärts-Algorithmus löst das Dekodierungsproblem. Er benutzt dazu dynamische Programmierung: Die Variablen $\beta_t(i) = P(o_{t+1} o_{t+2} \dots o_{T}|q_t = s_i, \lambda)$ geben die Wahrscheinlichkeit an, dass die Sequenz $o_{t+1} o_{t+2} \dots o_{T}$ beobachtet werden wird, gegeben das HMM $\lambda$ und den Startzustand $s_i$.

Forward-Backward Algorithm

Der Forward-Backward Algorithmus berechnet für jeden Zeitpunkt die Wahrscheinlichkeitsverteilung der Zustände. Dafür glättet er die Werte des Vorwärts- und des Rückwärts-Algorithmus: \[\gamma_t(i) = \frac{\alpha_t(i) \beta_t(i)}{P(O|\lambda)}\] Er findet jedoch nicht die wahrscheinlichste Zustandssequenz.

Viterbi-Algorithmus

Löst P2:
Siehe How to apply the Viterbi algorithm

Baum-Welch-Algorithmus

Löst P3: Gegeben sei eine Trainingssequenz $O_{\text{train}}$ und ein Modell \[\lambda = \{S, V, A, B, \Pi\}\] Gesucht ist ein Modell \[\bar \lambda = \text{arg max}_{\bar \lambda = \{S, V, \bar A, \bar B, \bar Pi\}} P(O_{\text{train}}|\lambda)\] Der Baum-Welch-Algorithmus geht wie folgt vor:

Bestimme $P(O_{\text{train}} | \lambda)$
Schätze ein besseres Modell $\bar \lambda$: TODO - Genauer! (Folie 31 - 36)

Iteriere diese Schritte so lange, bis ein lokales Maximum gefunden wurde.

Ergodisches Modell

Unter dem ergodischen Modell versteht man im Kontext von HMMs die vollverbundene Topologie inclusive Schleifen.

Bakis-Modell (Links-nach-Rechts-Modell)

Unter dem Bakis-Modell versteht man im Kontext von HMMs eine Links-nach-Rechts Topologie, bei der maximal ein Zustand übersprungen werden kann. Das bedeutet, es gibt eine Ordnung über den Zuständen. Von einem Zustand $i$ kommt man in die Zustände $i, i+1, i+2$.

Die drei Probleme von HMMs sind

P1 - Evaluierungsproblem: Wie wahrscheinlich ist eine Sequenz $\bf{o} = o_1 o_2 \dots o_T$ gegeben ein HMM $\lambda$, also $P(\bf{o}|\lambda)$.
P2 - Dekodierungsproblem: Finden der wahrscheinlichsten Zustandssequenz $s_1, \dots, s_T$, gegeben eine Sequenz von Beobachtungen $\bf{o} = o_1 o_2 \dots o_T$.
P3 - Lernproblem: Optimieren der Modellparameter

Anwendungen:

Gestenerkennung
Phonem-Erkennung

Markov Logik Netze

Slides: MLI_11-MLN_slides1

Markov Logik Netze sind Sammlungen von Tupeln aus Gewichten $w_i$ und prädikatenlogischen Formeln. Die Idee hinter Markov Logik Netzen ist ein aufweichen der harten Bedingungen der Prädikatenlogik. Eine prädikatenlogische Formel ist entweder wahr oder falsch. Eine Formel in MLNs kann auch “meistens” erfüllt sein. Das wird durch das Gewicht repräsentiert.

Markov Logik Netze (MLN): Ein Markov Logik Netz ist ein Menge aus Tupeln $L = (F_i, w_i)$, wobei $F_i$ eine Formel der Prädikatenlogik erster Ordnung und $w_i \in \mathbb{R}$ ein Gewicht ist. Ein MLN ist eine Schablone für ein MRF.
Markov Random Field (Markov Netzwerk, MRF): Ein MRF ist ein ungerichtetes Probabilistisches Grafisches Modell.
MRFs sind zur Modellierung von Korrelation geeignet.
Verbundwahrscheinlichkeit in MLNs: $P(x) = \frac{1}{Z} \exp(\sum_{i} w_i f_i(x))$ wobei $f_i$ das $i$-te Feature und $w_i$ ein Gewicht ist. Beispielsweise könnte \[f_i(x) = f_i(\text{smoking}, \text{cancer}) = \begin{cases}1 &\text{if } \neg \text{smoking} \lor \text{cancer}\\ 0 &\text{otherwise}\end{cases}\] gelten.
Inferenz in MLNs: MAP: \[\begin{align}\text{arg max}_y P(y | x) &= \frac{1}{Z} \exp(\sum_{i} w_i n_i(x, y))\\ &= \sum_{i} w_i n_i(x, y) \end{align}\]

Siehe auch:

Matthew Richardson, Pedro Domingos: Markov logic networks
Pedro Domingos, Matthew Richardson: Markov Logic: A Unifying Framework for Statistical Relational Learning, 2007.
Coursera: Probabilistic Graphical Models
Pedro Domingos: Unifying Logical and Statistical AI, September 2009.
Software: Alchemy
YouTube: 11 4 M4 Markov Logic Formalism 11 39

Evolutionäre Algorithmen

Slides: MLI_12_EvolutionaereAlgorithmen_slides1.pdf

Siehe auch:

[Mit97]
DEAP wenn du es ausprobieren willst.
Difference between genetic algorithms and evolution strategies?

Individuum

Eine mögliche Hypothese

Population (Generation)

Hypothesenmenge

Erzeugung von Nachkommen

Generierung neuer Hypothesen durch Rekombination und Mutation

Fitness-Funktion

Die Fitness-Funktion ist das zu optimierende Kriterium. Sie beschreibt die Güte einer Hypothese.

Selektion

Auswahl der Hypothesen, welche die beste Problemlösung erzeugen.

Evolutionäre Strategien

Das Wissen wird durch reele Zahlen und Vektoren repräsentiert.

Genetische Programmierung

Das Wissen wird duch baumartige Strukturen repräsentiert.

Mutation

Unter Mutation versteht man die zufällige Änderung einzelner Gene. Beispiele:

Bit-Inversion: Zufällig Gleichverteilt pro Gen / Feste Anzahl, aber zufällige Gene
Translation: Verschieben von Teilsequenzen
Invertiertes Einfügen
Spezielle Mutationsoperatoren sind anwendungsspezifisch

Rekombination

Bei der Rekombination werden die Eigenschaften zweier Eltern gemischt. Dies kann Diskret passieren, wenn manche Gene von einem Elternteil übernommen werden und andere vom anderen Elternteil. Alternativ kann die Rekombination auch durch intermediäre Rekombination passieren. Das bedeutet, das ein Gen gemittelt wird.

Grundalgorithmus:

Fitness-Function f
Population p

while f(p) ≠ optimal:
    p_parents ← selection(p)
    p_children ← generate_children(p_parents)
    p ← p_parents + p_children
    fitness ← f(p)
    p ← selection_kill(p, fitness)

Probleme:

Genetischer Drift: Manche Individuen vermehren sich zufällig mehr als andere. Diese sind nicht unbedingt besser für das Problem geeignet.
Crowding, Ausreißerproblem: Fitte Individuen dominieren die Population. Das ist ein Problem wegen lokaler Maxima.

Mating:

Inselmodell: Die Evolution läuft weitgehend getrennt. Es passiert nur vereinzelt, dass Individuen der Inseln ausgetauscht werden.
Nachbarschaftsmodell: Nachkommen dürfen nur von Individuen erzeugt werden, die in ihrer Nachbarschaft die beste Fitness besitzen
Globales Modell: Alle dürfen sich mit allen verbinden.

Evolution:

Lamark’sche Evolution: Die Individuen ändern sich nach der Erzeugung. Sie lernen also. Dabei wird der Genotyp verändert und auch vererbt.
Baldwin’sche Evolution: Die Individuen ändern sich nach der Erzeugung, aber der Genotyp bleibt gleich
Hybride Verfahren: Es gibt sich verändernde und gleich bleibende Phänotypen.

Anwendungen:

Traveling Salesman
Flugplanoptimierung
Mischung von Kaffesorten
Cybermotten: Motten müssen optimales Muster finden, um sich vor einer Fläche weißen Rauschens zu verbergen.
Snakebot (Ivan Tanev) [Pro06]

Deduktives Lernen

Slides: MLI_13_DeduktivesLernen_slides1.pdf

Siehe auch: Formale Systeme

Modus Ponens

\[\frac{A, A \rightarrow B}{B}\]

Erklärungsbasiertes Lernen (EBL, Explanation Based Learning by [Mit97])

The key insight behind explanation-based generalization is that it is possible to form a justified generalization of a single positive training example provided the learning system is endowed with some explanatory capabilitie. In particular, the system must be able to explain to itself why the training example is an example of the concept under study. Thus, the generalizer is presumed to possess a definition of the concept under study as well as domain knowledge for constructing the required explanation.

Explanation Based Generalization (EBG)

EBG ist ein Prozess, bei dem implizites Wissen in explizites umgewandelt wird. EBG geht wie folgt vor:

Explain: Finden einer Erklärung, warum das Beispiel die Definition des Zielkonzepts erfüllt. Dies ist einfaches Anwenden des Modus Ponens.
Generalize: Generalisieren der Erklärung; bestimme also hinreichende Bedingungen unter denen die gefundene Erklärungsstruktur gültig ist.

Bei der EBG werden also Makro-Operatoren erzeugt. Ein Beispiel für Software welche EBG benutzt ist STRIPS.

KBANN (Knowledge-Based Artificial Neural Networks)

KBANN ist ein hybrides Verfahren. Die Idee ist ein neuronales Netz geschickt zu konstruieren. Dieses wird dann wie gewohnt mit Gradient Descent durch Trainingsbeispiele verfeinert. Der Algorithmus gibt eine Netzarchtiktur vor:

Dabei wird pro Instanzattribut ein Netz-Input verwendet. Für jede Klausel wird ein Neuron hinzugefügt.
Dieses ist mit dem Instanzattribut durch das Gewicht $w$ verbunden wenn es nicht negiert ist, sonst durch das Gewicht $-w$.
Der Schwellwert der Aktivierungsfunktion wird auf $-(n- 0.5)w$ gesetzt, wobei $n$ die Anzahl der nicht-negierten Bedingungsteile ist.
Verbinde die restlichen Neuronen von Schicht $i$ mit Schicht $i+1$ indem zufällige kleine Gewichte gesetzt werden.

Angewendet werden kann KBANN:

Lernen von physikalischen Objektklassen
Erkennung von biologischen Konzepten in DNS-Sequenzen

Unüberwachte Lernverfahren

Slides: MLI_14_UnueberwachtesLernen_slides1.pdf

$k$-means Clustering

Der $k$-means Clustering Algorithmus finden $k$ Cluster in einem Datensatz. Dabei ist $k \in \mathbb{N}_{\geq 1}$ vom Benutzer zu wählen. Zuerst initialisert $k$-means die Zentroiden, also zentrale Punkte für Cluster, zufällig. Dann geht $k$-means geht iterativ vor:

Weise jeden Datenpunkt seinem nächsten Cluster zu.
Verschiebe die $k$ Zentroide in ihr Clusterzentrum

Siehe auch: Interaktives Beispiel

Fuzzy $k$-means

Im Gegensatz zum $k$-means Algorithmus, wo jeder Datenpunkt in genau einem Cluster ist, weißt der Fuzzy $k$-means Algorithmus jedem Datenpunkte eine Zugehörigkeitswahrscheinlichkeit zu. Je weiter der Datenpunkt vom Zentroid entfernt ist, desto unwahrscheinlicher wird die Zugehörigkeit. Die Cluster-Zugehörigkeit des Datenpunktes $x_i$ zum Cluster $c_j$ kann als Wahrscheinlichkeit in Abhängigkeit der Distanz \[d_{ij} = |x_i - z_j|^2\] zum Zentroiden $z_j$ ausgedrückt werden: \[P(c_j | x_i) = \frac{(\frac{1}{d_{ij}})^{\frac{1}{b-1}}}{\sum_{r=1}^k (\frac{1}{d_{ir}})^{\frac{1}{b-1}}}\] wobei $b \in \mathbb{R}_{\geq 1}$ ein frei zu wählender Parameter ist. Die Zentroide werden dann wie folgt neu berechnet: \[z_j = \frac{\sum_{i=1}^n [P(z_j|x_i)]^b \cdot x_i}{\sum_{i=1}^n [P(z_j | x_i)]^b}\]

Hierarchisches Clustern

Die Idee des hierarchischen Clusterns ist die iterative Vereinigung von Clustern zu größeren Clustern. Ergebisse können durch ein Dendrogramm beschrieben werden. Anwendung: Einordnung von Schrauben in ein Ordnungssystem

Agglomerative Hierarchical Clustering (AHC)

AHC ist ein hierarchisches Clusteringverfahren. Dabei ist ein Clusterdistanz-Schwellwert $t \in \mathbb{R}$ und eine minimale Cluster-Anzahl $k \in \mathbb{N}$ zu wählen. Auch ein Distanzmaß für Cluster (nearest neighbor, farest neighor, mean distance, ...) ist als Hyperparameter zu wählen. Dann geht AHC wie folgt vor:


c ← k  # Minimale Anzahl an Clustern
c' ← n  # Anzahl der Datenpunkte

# Weise jedem Punkt sein eigenes Clusterzentrum zu
for i in range(1, n):
    D_i ← {x_i}

# Vereinige Clusterzentren
do:
    c' := c' -1
    find closest clusters D_i, D_j
    if d(D_i, D_j) ⩽ t:
        merge(D_i, Dj)
    else:
        break
until c = c'

Begriffliche Ballung

Bei Algorithmen der Begrifflichen Ballung werden Konzeptbeschreibungen generiert.

COBWEB

Cobweb ist ein Algorithmus zur begrifflichen Ballung. Er lernt durch inkrementelles Aufbauen eines Strukturbaumes. Dabei sind nominale Attribute gestattet. Dabei wird ein Datenpunkt $x_i$ zum Cluster $c_j$ geclustert, wenn man die Attributwerte von $x_i$ durch die Kentniss von $c_j$ gut vorhersagen kann ($P(x_i | c_j)$, predictability) und zugleich der Cluster gut vorhergesagt werden kann, wenn die Attributwerte gegeben sind (\(P(c_j|x_i), predictiveness). Es soll also in inter-Klassenähnlichkeit minimiert und die intra-Klassenähnlichkeit maximimiert werden. Dafür wird die Category Utility verwendet:

\[\text{CU} = \sum_{k=1}^K \sum_{i=1}^I \sum_{j=1}^{J(i)} P(A_i = V_{ij}) \cdot P(A_i = V_ij | C_k) \cdot P(C_k | A_i = V_{ij})\]

Dabei gilt:

$K$: Anzahl der Cluster
$I$: Anzahl der Attribute
$J(i)$: Anzahl der Attributwerte des $i$-ten Attributs
$V_{ji}$: $j$-ter möglicher Wert für Attribut $i$
$P(A_i = V_ij | C_k)$: Predictability
$P(C_k | A_i = V_{ij}$: Predictiveness

Anwendung: Interpretation von EMGs

Prüfungsfragen

Was ist Induktives Lernen?
→ Eine große Menge an Beispielen wird gegeben. Der Lerner muss selbst das Konzept herausfinden.
Was ist Deduktives Lernen?
→ Fakten werden gegeben. Der lernende bekommt das allgemeine Konzept gesagt und muss nur logische Schlussfolgerungen machen.
SVMs
- Wie funktioniert SRM bei SVMs?
  → Dualität zwischen Feature- und Hypothesenraum: Radius der Hyperkugel wird minimiert.
- Warum lernen SVMs "korrekt"?
  → Es gibt ein Theorem (TODO: Welches?) das besagt, dass die VC-Dimension eines Klassifiers, welcher Datenpunkte im $n$-Dimensionalen Raum innerhalb einer Kugel mit Radius $D$ durch eine Hyperebene mit mindestens Abstand $\Delta$ trennen will, durch $(\frac{D}{\Delta})^2$ beschränkt ist. Die SVM minimiert genau diesen Quotienten, da sie den Margin maximiert. Alternativ: Erklärung durch Strukturierung des Hypothesenraumes (TODO).
Reinforcement Learning
- Wie lautet die Bellman-Gleichung?
  → $Q(s, a) = r + \gamma \max_{a'} Q(s', a')$ wobei $\gamma$ ein Diskontierungsfaktor ist, $s'$ der Zustand in den man kommt, wenn man $a$ ausführt und $r$ der Reward nach ausführen von $a$ in $s$ ist.
- Was ist Value Iteration und wie lautet die Formel?
  → Schätzen der Value-Funktion durch iteratives anwenden von $\hat{V}^*(s_t) \leftarrow r_t + \gamma \hat{V}^*(s_{t+1})$
- Was sind Eligibility Traces im Kontext von Reinforcement Learning?
  → Siehe oben
- Wie funktioniert Q-Learning?
  → Siehe Abschnitt Q-Learning
Evolutionäre Algorithmen: Was ist wichtig?
- Population / Individuen: Wie Individuen darstellen
  → Durch Gene (Attribute), z.B. als Bitstring
- Gegebener Ablauf (Wahl der Eltern, Generierung der Individuen)
- Wie kann man Kombinieren?
  → vgl. Rekombination
- Fitness Function
- Was sind die wichtigsten Elemente von evolutionären Algorithmen?
  → Mutation, Rekombination, Fittness-Funktion, Selektion
- Was ist Landmarksche / Baldwinsche Evolution?
Wie lautet die Fehlerabschätzung von Vapnik?
→ Siehe Abschätzung des realen Fehlers durch den empirischen Fehler und die VC-Dimension.
Was versteht man unter Cascade Correlation?
→ YouTube (4:05 min)
Welche übwerwachten Lernverfahren gibt es?
→ Neuronale Netze, SVMs
Wie funktioniert Inferenz in Markov Logik Netzen?
→ Siehe oben
Wie wird die Verbundwahrscheinlichkeit / Weltwahrscheinlichkeit in Markov Logik Netzen berechnet?
→ Siehe oben
Was ist Dynamic Decay Adjustment (DDA)?
→ Siehe oben
Was ist erklärungsbasierte Generalisierung (EBG)?
→ Der Agent lernt keine neuen Konzepte, aber er lernt über Verbindungen bekannter Konzepte.
Wie lautet die Formel für Entropie / Information Gain?
→ $\text{Entropie} = - \sum_{i} p_i \log p_i$ und $KL(P, Q) = \sum_{x \in X} P(x) \cdot \log \frac{P(x)}{Q(x)}$
Was ist Cobweb?
→ Siehe Unsupervised Learning

Material und Links

Vorlesungswebsite
Übungswebsite
StackExchange
- What is the difference between concept learning and classification?
- What is the difference between a (dynamic) Bayes network and a HMM?
Zusammenfassung der Vorlesung ML 2
Udacity
- Knowledge-Based AI: Cognitive Systems: Unter anderem gibt es eine Lektion zu Explanation-Based Learning (erklärungsbasierte Generalisierung)

Literatur

[Mit97] T. Mitchell. Machine Learning. McGraw-Hill, 1997.
[Dar09] A. Darwiche. Modeling and reasoning with Bayesian networks. Cambridge University Press, Cambridge [u.a.], 2009.
[Ber95] M. Berthold and J. Diamond. Boosting the Performance of RBF Networks with Dynamic Decay Adjustment. Advances in Neural Information Processing, 1995. [Online]
[Pro06] Prokopenko, Mikhail and Gerasimov, Vadim and Tanev, Ivan. Evolving Spatiotemporal Coordination in a Modular Robotic System. Springer, 2006.

Übungsbetrieb

Es gibt keine Übungsblätter, keine Übungen, keine Tutorien und keine Bonuspunkte.

Termine und Klausurablauf

Datum: Mündliche Prüfung (in Zukunft schriftlich)
Ort: nach Absprache
Zeit: 15 min
Übungsschein: gibt es nicht
Bonuspunkte: gibt es nicht
Ergebnisse: werden ca. 5 - 10 min. nach der mündlichen Prüfung gesagt
Erlaubte Hilfsmittel: keine

Wo ist Hörsaal 9 am KIT?

info@martin-thoma.de (Martin Thoma) — Mon, 02 Nov 2015 21:33:00 +0100

Dieser Artikel ist für all die verzweifelten Studienanfänger am KIT, die den Hörsaal 9 (HS 9) suchen.

Der HS 9 ist in Gebäude 20.40, dem Architektengebäude. Es ist im ersten Stockwerk. Da muss man die große Treppe des Haupteingangs hoch, nach rechts abbiegen und um die Ecke laufen.

Sinnigerweise ist Hörsaal 37 (HS 37) im Erdgeschoss…

&ast;sigh&ast; da hätte ich wirklich gerne Google Indoor Maps. Oder zumindest im Ergeschoss einen Plan, der Pläne der Stockwerke beinhaltet. Am besten noch eine alphabetisch sortierte Liste der Räume mit Verweis auf die Etage…

(Randnotiz: Falls sich jemand über diesen Mini-Artikel wundert: Ich habe diese Hörsäle schon mehrfach gesucht. Letzens wegen Statistik, wo das so wunderbar präzise auf der Vorlesungsseite steht… nun kann man es hoffentlich bald googeln, auch wenn ich nicht dabei unterstützt wurde, die Pläne auf Google Indoor Maps zu stellen. Irgendwas von “Datenschutz” wurde gesagt…)

Computergrafik - Klausur

info@martin-thoma.de (Martin Thoma) — Fri, 23 Oct 2015 11:30:00 +0200

Dieser Artikel beschäftigt sich mit der Vorlesung „Computergrafik“ am KIT. Er dient als Prüfungsvorbereitung. Ich habe die Vorlesungen bei Herrn Prof. Dr. Ing. Carsten Dachsbacher im Wintersemester 2015/2016 gehört. Der Artikel ist noch am Entstehen.

Behandelter Stoff

Vorlesung

Datum	Kapitel	Inhalt
20.10.2015	Einleitung	Viele nette Bilder und Beispiele, wenig Inhalt
22.10.2015	Farbe, Darstellung & Perzeption	Weber-Fechner-Gesetz, Abtasttheorem, Dynamikumfang, Farbwahrnehmung im menschlichen Auge; Gammakorrektur
26.10.2015	Übung	gdb (bt - backtrace; p - print)
26.10.2015		Metamerismus: Unterschiedliche Spektren können gleichen Farbeindruck erwecken Addiere/subtrahiere Farbmischung CMY / CMYK / RGB / HSV / XYZ Weber-Fechner-Gesetz: 2% heller
29.10.2015	Ray-Tracing: Kapitel 2	Menschen nehmen Kontrastintensität und Luminenz besser als Chrominanz war. Das ermöglicht Kompression. clear-type / subpixel Darstellung Jaggies: Unerwünschter Treppenstufen-Effekt bei Rasterisierung von Strecken Kamera: Position (x,y,z), Blickrichtung (zu einem Punkt mit Koordinaten (x,y,z)) und up-Vektor Skalarprodukt, Kreuzprodukt Baryzentrische Koordinaten,
02.11.2015	Übung	Rasterisierung von Linien (Implizite Darstellung) zbuffer: Tiefe des "nächsten" Polygons pro Pixel wird gespeichert Konsistenzregeln, z.B. wenn man zwei benachbarte farbige Dreiecke mit Diagonale hat und man die Farbe des Pixels berechnen muss, durch den beide Dreiecke teilweise gehen (37 Fälle; Katalog von Microsoft; macht die Hardware)
26.11.2015	?	Sphere Mapping; Vorfilterung von Environment Maps
10.12.2015	Räumliche Datenstrukturen	BSP-Baum, kD-Baum
14.01.2016	OpenGL
18.01.2016	Übung	Koordinatensystem-Pipeline, Shader
19.01.2016	Erzeugung von Landschaften	Rotes / Rosa Rauschane, Lattice Value Noise, Perlin-Noise

Folien

Bilder, Farbe, Perzeption

Slide: 01_ Bilder, Farbe, Perzeption - Teil1.pdf

Frame Buffer: Speichert Bilder zur direkten wiedergabe auf dem Bildschirm.
Ditherhing: TODO
Dynamikumfang: Der Dynamikumfang beschreibt den erreichbaren Kontrast eines Wiedergabegrätes (Bildschirm, Beamer): \[R_d = \frac{I_{\text{max} + k}}{I_{\text{min}} + k}\] TODO: Was ist $k$? Was ist $I_{max} / I_{min}$?
Gamma-Korrektur: TODO
Transferfunktion: TODO

Slide: 01_ Bilder, Farbe, Perzeption - Teil2.pdf

Additive Farmbischung: Grundfarben: Rot, Grün, Blau
Anwendung: Bildschirm
Subtraktive Farbmischung: Grundfarben: Cyan, Magenta, Gelb
Anwendung: Drucker
Graßmansche Gesetze: Jeder Farbeindruck kann mit 3 Grundgrößen beschrieben werden.

RGB-Farbraum
HSV-Farbraum
CIE Color Matching Functions
XYZ Color Space
Chromatizität
xyY Farbraum
Weber-Fechner-Gesetz
Machsche Streifen / Bandeffekte
Hermann-Gitter / Laterale Hemmung
Windows clear type / Subpixel

Raytracing

Side: 02_ Raytracing (enthalt Abtastung aus Kapitel 1).pdf

Nyquist-Shannon-Abtasttheorem
Vektoren, Ortsvektoren, Skalarprodukt
Parametrisierte Geraden- und Ebenendarstellung
Baryzentrische Koordinaten
Strahl-Kugel-Schnitt
Spekulare Reflektion
Diffuse (Lambertsche) Reflektion
BRDF - Bidirectional Reflectance Distribution Function
Phong Beleuchtungsmodell
Snellsches Brechungsgesetz
Fresnel-Effekt
Anti-Aliasing Strategien: Uniformes Supersampling, Adaptives Supersampling, Stochastisches Supersampling
Schattenstrahlen
Bewegungs- und Tiefenunschärfe
Imperfekte Spiegelung und Transmission

Räumliche Datenstrukturen

Slide: 05_ Raumliche Datenstrukturen.pdf (10.12.2015)

Hüllkörper
- Axis-Aligned Bounding Boxes (AABB)
- Bounding Volume Hierachies (BVH)
Reguläre Gitter
Oktalbaum (Octree)
kD-Baum

BSP-Baum: TODO
Surface Area Heuristics (SAH): Schätzfunktion für die Oberfläche eines Objekts (TODO?).

Rasterisierung, Clipping und Projektionstransformationen

Side: 06_ Rasterisierung, Clipping und Projektionstransformationen.pdf

Tiefenpuffer, Z-Buffer
Clipping
Sutherland-Hodgeman Polygon Clipping

19.01.2016

Perlin-Noise: Zufallszahlen-Pool + Hash + Permutation
Oktave: Sammlung von Noise-Funktionen

Prüfungsfragen

Kommt noch… spätestens wenn die Klausur naht.

Übungen

Blatt 1

Das Framework bekommt man ohne VM unter Ubuntu 15.04 nach der Installtion folgender Pakete (vielleicht) zum laufen:

$ sudo apt-get install cmake xorg-dev libglu1-mesa-dev freeglut3 freeglut3-dev libglew1.5 libglew1.5-dev libglu1-mesa libglu1-mesa-dev libgl1-mesa-glx libgl1-mesa-dev libglfw3

Wenn ihr den Fehler

error adding symbols: DSO missing from command line ubuntu

bekommt, dann solltet ihr einfach die obigen Pakete installieren, den build-Ordner löschen und es neu versuchen.

Color cube
Gravity field
Temperature of a black body

Außerdem:

$ pacman -Syy

ausführen. Dann bekommt man auch nicht mehr 404er wenn man mit

$ pacman -S vim

vim installieren will.

In der VM sollte unter Settings → System → Acceleration die Option “Enable VT-x/AMD-V” aktiviert sein. Zusätzlich sollte im BIOS des Host-Systems (also von eurem Rechner) die “Intel Virtualization Technology” aktiviert sein. (Man Laptop hat das nicht - bei mir funktionieren die Beispiele in der VM aber auch nicht :-/)

Blatt 6

glm::gtx::transform

Material und Links

Siehe auch

Literatur

P. Shirley, S. Marschner: Fundamentals of Computer Graphics, 3rd Edition
→ Kapitel 3-4, Kapitel 7-9, Kapitel 11-12 (Data Structures for Graphics)

Übungsbetrieb

Es gibt Übungsblätter und Übungen, aber keine Tutorien und keine Bonuspunkte.

Um das Modul zu bestehen wird der Übungsschein benötigt. Für den Übungsschein benötigt man 60% der Punkte der Übungsblätter. Die Übungsblätter werden über submit.ivd.kit.edu eingereicht. Die Übungsblätter erscheinen alle 2 Wochen. Es gibt also min. 6 Übungsblätter und min. 120 Punkte. Die Deadline ist Montag, 11:00.

Termine und Klausurablauf

Datum: Mittwoch, der 09.03.2015 von 14:00 Uhr (Quelle: informatik.kit.edu)

08.02.2016: Die Klausur-Anmeldung wird freigeschalten
04.03.2016: Anmeldeschluss
06.03.2016: Abmeldeschluss

Ort: 10.21 (Daimler und Benz)
Punkte: ?
Zeit: ? min
Punkteverteilung: ?
Bestehensgrenze: ?
Übungsschein: ?
Bonuspunkte: ?
Ergebnisse: ?
Einsicht: Noch nicht bekannt (Stand: 23.10.2015)
Erlaubte Hilfsmittel: keine

Rename Script

info@martin-thoma.de (Martin Thoma) — Wed, 16 Sep 2015 16:15:00 +0200

Once in a while I have to bulk rename files. For example, when I have holiday photos. The following script helps me to do so:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

"""Batch-Rename files in a folder according to some rules."""

import os


def rename(rename, starts=None, ends=None, test=False):
    """
    Rename all files which end with `ends` and start with `starts` to
    `rename-[Number]`.
    """
    files = [f for f in os.listdir('.') if os.path.isfile(f)]
    # Sort files by last modification
    files.sort(key=lambda x: os.stat(x).st_mtime)
    if starts is not None:
        files = [f for f in files if f.lower().startswith(starts.lower())]
    if ends is not None:
        files = [f for f in files if f.lower().endswith(ends.lower())]
    for i, f in enumerate(files, start=1):
        filename, ext = os.path.splitext(f)
        ext = ext.lower()
        new_name = "%s-%i%s" % (rename, i, ext)
        if test:
            print("{0:<40}-> {1:<20}".format(f, new_name))
        else:
            os.rename(f, new_name)


def get_parser():
    from argparse import ArgumentParser, ArgumentDefaultsHelpFormatter
    parser = ArgumentParser(description=__doc__,
                            formatter_class=ArgumentDefaultsHelpFormatter)
    parser.add_argument("-n", "--name",
                        dest="name",
                        help="new name",
                        required=True,
                        metavar="NAME")
    parser.add_argument("-s", "--starts",
                        dest="starts",
                        help="Get files which start with this string",
                        default=None,
                        metavar="STARTS")
    parser.add_argument("-e", "--ends",
                        dest="ends",
                        help="Get files which end with this string",
                        default=None,
                        metavar="ENDS")
    parser.add_argument("-t", "--test",
                        action="store_true",
                        dest="test",
                        default=False,
                        help="don't change anything, just show what would be "
                             "done")
    return parser


if __name__ == '__main__':
    args = get_parser().parse_args()
    rename(args.name, starts=args.starts, ends=args.ends, test=args.test)

Languages for Back Ends

info@martin-thoma.de (Martin Thoma) — Wed, 24 Jun 2015 12:43:00 +0200

What programming language would I use for the back end of a big, new project in a startup which wants to offer a web service? Sure, on the client side there is pretty much only JavaScript (including variants like CoffeeScript and TypeScript) in combination with HTML and CSS. I’ve used MySQL and Redis databases and I’m quite happy with that. But the choice for the server side is not that easy. I’ve been using PHP for quite a while now, because it was the cheapest and easiest choice when I started programming. But things have changed (and I have more money, so I don’t have to take the super cheap hosting services). Although my experience with web projects is very limited, I want to share a few thoughts.

Definitions: Back End and Security

Just for clarification: I am only talking about the back end. A back end is the data access layer which manages requests comming to the server. It needs to server many requests (> 100 requests/second) fast (< 300 ms in average). It should not execute computationally heavy jobs which can be pre-computed or do not need to be displayed instantly to the client. This can be done by another system which does not need to be programmed in the same language. The back end does also not deal with presentation to the user. This is what the front end does. However, you should have more than a good idea in which form the front end gets the data. The cleanest approach I’ve seen so far is a pure RESTful API for all interactions between front end and back end.

The backend language should also make it easy to validate / sanitize input data, connect with databases, store/get stuff on/from the file system.

In the following, I will write that some languages are “secure” or “not secure”. This does not mean that you can / cannot write code which is secure. It means that the compiler (or other widespread tools) give you guarantees about bugs in your code. For example, C is a very insecure language as the compiler does no bounds checking. The types of errors which can be detected by automatic tools (without further testing) are:

Syntax errors,
Out of bounds (reading),
buffer overflow (not checked in C/C++, but not possible in Java (source)),
unused variables (which might indicate other problems; at least code smell),
type problems: This is a bit fuzzy, as you can write stringly typed code (see New Programming Jargon) in probably every language, but in some languages it is more common than in others. Some languages also make it easier to use the type system to detect errors. For example, PHP is very insecure in this sense as 123 == "123ab", Python is a bit more secure, but you can return whatever you want, Java is much more secure. Haskell is even more secure in this sense, as it has real functions (without side effects, checked by the compiler). See What can Haskell’s type system do that Java’s can’t and vice versa? for more.

There are also some errors which can be detected at runtime. The handling of those runtime errors differs from language to language. For example, C and C++ fails silently (e.g. this question). This is bad. For example, there are some silent out-of-bounds errors in C / C++ where Rust would fail loud (I think Heartbleed is one example; see Would Rust have prevented Heartbleed? Another look if you’re interested in that specific example).

Of course, all of those problems can be detected with good testing. But the more is done automatically, the less can go wrong when you don’t write (good) tests.

Java

Java is an object-oriented language which runs on the Java Virtual Machine. Java is probably the most used language for big business websites. Why is that the case?

Java is old: It first appeared in 1995.
- Java is taught at many universities and many people know at least a little bit of Java. So companies don’t have problems finding developers. At least that might be the impression of people who don’t realize that there is a big differencee between people saying they know Java and developers who can actually work with it.
- I guess the Java ecosystem is pretty mature:
  - eclipse, IntelliJ IDEA and Netbeans as IDEs,
  - Jenkins for continuous integration,
  - GlassFish,
  - Apache Ant/Apache Maven or Gradle for automatic building,
  - JUnit, Mockito, Powermock for automatic unit tests,
  - log4J and log4J 2 for logging,
  - Apache JMeter for load testing
  - Jersey for RESTful Web services,
  - Apache Tomcat / WildFly (former JBoss): application server / web server / servlet container
  - Grizzly / Jetty: Web server
  - FindBugs, SonarQube for code quality / static code analysis
  - Hibernate for ORM,
  - OSGi: Apache Felix / Equinox - see 10min clip for a high-level explanation of OSGi,
  - Frameworks like Spring, JSF, JSP, Apache Struts 2, Apache Wicket
Java is developed by Oracle. Hence you can make contracts with Oracle to get support when things don’t work.

That was what we have on the positive side. What is not so good about Java?

VERY clumsy syntax. This is more than just a inconvenience. You have to type a lot to get things done which makes you slow. Of course, you can (and need to) use autocompletion, but it is still a lot to read. That makes maintaining the code a mess.
Tools are hard to get to work.
Unnecessary super-abstract constructs used for eventually never happening future extensions (see Geek-and-poke.com).
A bit more secure than C/C++ as you cannot access out-of-bound arrays, you don’t have pointers. So buffer overflows are almost impossible in Java (see SO for more details). However, you buy this security with much less easy syntax and you don’t get as much security as would be possible with just a bit more effort. See rust for more details.
Speed and memory usage: Again, Java might be better in speed than many other languages, but not as good as some others are. And Java seems to need A LOT of memory. However, I am not too sure if that is really a problem. (see Surprise! Java is fastest for server-side Web apps)

See also:

Is Java a Compiled or an interpreted programming language?: The short answer is no. But Java guys don’t like to hear that ☺
Why do I hear about so many Java insecurities? Are other languages more secure?: The short answer is no, thinking about C/C++.
Security of JVM for Server
C++ performance vs. Java/C#

JavaScript: Node.js

Node.js is a runtime environment which was initially released in 2009 and became quite popular since then. Node.js is asynchronous, event-driven and scalable. Node.js applications are written in JavaScript and hence have all the advantages of JavaScript:

They profit from heavy development in JavaScript engines / JIT compilers like V8.
The syntax is flexible and light-weight.
Just like Java, JavaScript first appeared in 1995. So the language itself is old and stable.
- A lot of developers know at least a little bit of JavaScript.
- The ecosystem is mature.
  - npm and bower for package management
  - Backbone.js / AngularJS for MVP / MVC.
  - Unit.js for unit testing.
  - Grunt as a task runner.
  - Sequelize as an ORM.
  - Karma: Test runner
  - expressjs: web application framework
Lots of easy tutorials

What is still to say?

Node is FAST and scalable! (see Performance Comparison Between Node.js and Java EE)
JavaScript is very insecure. Even simple syntax error will only get revealed when they are actually executed. So Unit testing is very important.
Node.js is used by LinkedIn, Yahoo!, Uber, PayPal (source)
There are quite a few people moving from Node.js to Go (1, 2, 3, 4)

See also:

Go

Go is a statically-typed, compiled language developed by Google. It first appeared in 2009, so it is very young.

Go offers the basic tools you need for web development:
- martini/ Gin Gonic: A web development framework
- mustache for templates
- gorm: ORM
Good tutorial and also some material for web development
Some tasks are much more complicated than they should be. Sorting, to name one example (see SO).
Go is different from some other languages, e.g. if you want a method to be public, the first character of the method name has to be capitalized. Or unused variables result in a compiler error.

See also:

C#

C# is a compiled, statically typed language (with dynamic features, see Understanding the Dynamic Keyword in C# 4) developed by Microsoft. It was publically announced in 2000. The initial release of its web appliction framework ASP.NET was in 2002.

The ecosystem seems to include:

nuget.org
IIS: Web server
Entity Framework: ORM
LINQ: SQL queries
Visual Studio: IDE
ASP.NET MVC Framework

But I don’t know enough about C# / ASP.NET to write something meaningful about it.

Coding Horror described why they use ASP.NET for StackOverflow and why he doesn’t recommend it for OpenSource projects (source). StackExchange also describes what they use (1, 2).

I see a big problem in the Microsoft-centric technology stack. ~~You have to use everything from them. (Almost) everything is closed source. If they discontinue the development or if they don't fix stuff which might be relevant for you, you're fucked.~~

This seems to change. Microsoft moved some important parts of their stack to GitHub (see dotnet.github.io). Most important seems to be that the compiler Roslyn is licensed under an Apache License. But there is also ASP.NET, the Entity Framework, and the .NET runtime. The Visual Studio Community Edition is not available for free (but only for Windows).

Python

Python is one of the oldest programming languages which are still in use. It first appeared in 1991. Python is dynamically typed, interpreted, object-oriented and includes functional programming features.

Although I use Python for many projects, I didn’t use it by now for a web project. So I might not know the important tools / frameworks. Please keep that in mind.

Ecosystem:
- pypi.python.org and pip: Package hosting and package management
- Sphinx: (Semi) automatic code documentation, e.g. the scipy docs are generated with Sphinx from Python code. This is one of the best documentations I have ever seen.
- Django/ Flask as frameworks
- pytest/nose for testing
- gevent: a coroutine-based Python networking library
- Tornado: Web server
Some Python people switch to Go (1, 2)
Many tutorials and often very good documentation:
Flask and Django work with PyPy (source). That might make them much faster.
Used by big players:
- Quora (source)
- Prezi, Pinterest, Instagram (source)
- Bitbucket, The Onion (source)

I think one of the main advantages of Python is that it is really easy to write code which is easy to read (because of docstrings, Pythons weird intendation semantics and very nice syntax) and quite hard to write unreadable code. I am sure I have a biased view regarding Python, but I am also sure a lot of people share this subjective impression.

PHP

PHP is a server-side scripting language which appeared first in 1995. It is dynamically typed.

Language inconsistencies are really bad with PHP - see also PHP: a fractal of bad design
The ecosystem is ok:
- PHPCI for continuus integration.
- Zend Framework / Symfony
- Smaller Frameworks like CakePHP and Code Igniter
- Drupal / Joomla / TYPO3 / WordPress
- PHPUnit for unit testing,
- Composer for package management and packagist.org to find packages
- cruisecontrol for Continuus Integration

A big advantage of PHP is that it is easy to learn. You can run PHP everywhere and hosting is cheap. Wikipedia makes use of PHP, so it is obviously possible to create systems which have HUGE numbers of requests and still work fine.

The right tool for the right job - by commitstrip.com

Hack

Hack is a programming language introduced in 2014 by Facebook. It is a PHP dialect. Key differences to PHP are:

Function arguments and return values can be annotated with types.
Hack does not support some language features which are supported by PHP (source). Which is good. For example, goto, variable variables, string incrementing, …

See also:

hacklang.org

Rust

Rust is a very safe language, but seems not to be ready for productive usage.

I am a big fan of Rust, but as it aims to be a better C++, it is probably a better fit for OS development, game engines, embedded systems, databases, complex desktop applications (Photoshop/Word/Chrome), etc. While Rust is quite expressive for a systems programming language, its banner features are the borrow checker (+ lifetimes, etc) and powerful static type system. Rust emphasizes zero-cost abstractions with compiler-enforced memory & thread safety. The popular web development languages are dynamically typed and interpreted, with an emphasis on rapid development, which is a very different niche than Rust claims to fill.

Source: news.ycombinator.com

See:

Others

Ruby with Rails: I know it is quite well-known and used by many people. But I don’t know Ruby enough to write anything meaningful. The Ruby syntax is similar to Python.
Scala seems to be noteworthy

Conclusion

Thinking about it that carefully, I see three languages which seem to be suitable for back ends for me:

Go: Fast and compiled
node.js: Good scalability
Python: It is the language I know best and of which I like the syntax best. Besides that, it has a very nice and clear syntax, good community-developed coding style standards and is very easy to read and well-documented.

Not suitable seem to be:

PHP: Because of the language inconsistencies which seem to make it pretty hard to make a reliable back end
C#: The technology stack is too Microsoft centered.
Java: Too clumsy syntax, too hard to get it work.

The other programming languages could be very good choices. I simply don’t know it. I am very curious if rust will be used for back ends. Hack is very young, let’s see if it will spread in a few years.

Credits

As I don’t have much experience with web development, I asked a few friends to have a look at the different parts of the article. They looked especially at plain wrong statements, if I named “all” the important frameworks / tools. They might not completely agree with the comparison to other language (after all, I wrote the article), but they helped me a lot to get things not too wrong:

Sören Liebich (@liebsoer) has several years of experience with Java web development and helped me to name the important tools / technologies used in the Java stack.
Henning Dieterichs helped me to fix some of the mistakes in the C# part and reminded me of Hack and the positive sides of PHP.
Stefan had a look at the PHP section.

Thank you!

Increase the maximum file upload size in PHP

info@martin-thoma.de (Martin Thoma) — Sun, 07 Jun 2015 09:13:00 +0200

Create a test.php with the following content to check your current maximum upload size.

<?php phpinfo(); ?>

Search your php.ini in the Apache folder:

$ find /etc -name 'php.ini'
/etc/php5/cli/php.ini
/etc/php5/apache2/php.ini

Edit it. Set upload_max_filesize and post_max_size to whatever you want.

Restart the server with

$ sudo service apache2 restart

3D Photospheres

info@martin-thoma.de (Martin Thoma) — Thu, 21 May 2015 16:23:00 +0200

I’ve been to New York City a few months ago. It was an interesting experience. Everything is so big, it is difficult to get a feeling for it if you haven’t seen it yourself. For example, the WTC Memorial is overwhelmingly large. I tried to capture that in a photograph with my Panasonic DMC-TZ41, but it just doesn’t look that large:

WTC Memorial

A photosphere captures that much better (it looks great on my Smartphone, but not nearly as well on the web): Android Photosphere

Optonaut goes one step further. This app is able to create 3D photospheres.

How it works - User perspective

You start the app, slowly turn around yourself to capture everything around you. This includes your feet and the sky, if you really want a complete photosphere. The recording process the same as for Android Photospheres, but probably with a slightly different interface:

Optograph recording process

When you want to watch it in 3D you have to have something that holds it in front of your face at the right distance. Cardboard is an easy and cheap possibility to do so:

Cardboard

And, of course, you can already share optographs:

Sharing Optographs

How it works - Developers perspective

Optonaut makes use of the camera and takes a lot of images. Those images are combined to a photosphere by a process called image stitching. This means you search points which seem to match to build one big image from many small images. A big problem is that your camera moves quite a lot in space and the environment changes. So there might not be any way to make a perfect match. Sensors like the compass and the accelerometer help to figure out at which direction you are currently pointing. Programs like hugin stitch images semi-automatically, but Optonaut works completely automatically.

From my experience with Android Photospheres, this works much better when the objects around you are far away. That means it doesn’t work in your room.

Then Optonaut uses a technique called stereoscopy. They show you different images for each eye to give you the impression of a 3D image. They also use a different photosphere techique to “mix images dynamically”. Optonaut claims that this does work for close objects (in contrast to the traditional photosphere apps like Google Photosphere, Photosynth or Microsoft ICE).

As they can figure out where you look at with the accelerometer / compass, they adjust the images accordingly for each eye to create a 3D impression:

Stereographic images produced by Optonaut

Now it gets very interesting. Usually, you would need two cameras next to each other to take two slightly shifted images. But Optonaut needs only one camera. They make use of the fact that you rotate around your axis to photograph your environment. Optonaut takes two images which are next to each other, match them, deskew them. Et voilà: You are able to get a 3D impression!

A side note: The 3D photosphere is stored as an archive of images.

The Optonaut Team

Optonaut has two founders: Emanuel Jöbstl and Johannes Schickling. I’ve had the luck to work with them in my first hackathon (CODEFEST8). We won in Karlsruhe with an App / web service with a fully automatic drivers logbook (logbookapp.de). Both of them are great developers. Emanuel created the (working!) Android app which connected to the car within a couple of hours and Johannes made a beatiful frontend in the same time.

Emanuel knows a lot about protocols, can understand mathematically complex algorithms and solve conceptually difficult problems. I guess he wrote the stitching algorithm.

I’ve learned a lot from Johannes about tools. He is a perfectionist. In situations where most people would say “hey, it works, lets drink some coffee” he is still working hard to get the best user experience, making the code more maintainable by refactoring it, improving the way how code is built / deployed. In a short-term project this costs probably more time than doing it quick-and-dirty, but on the medium/long run (e.g. more than 2 weeks) it saves a lot of work.

And Optonaut is such a long-term project. They are working on it for quite a while now, so I am sure the quality is pretty high. I am very curious if the Kickstarter campaign works as well as they hope and I cannot wait for the first people to share their optographs from amazing places!

Future of Optonaut

The Optonaut team plans to build a sharing site where you can show others amazing places. They want to completely redesign the App.

I guess the stitching algorithm is something where you can put endless effort in. It is an optimization problem where it is not completely clear what to optimize, so I guess this will continually be improved.

I want it! What do I need?

You need an Android / iOS smartphone. There seems to be a web-based viewer which works with Chrome/Firefox/Safari, too.

Getting a Feeling for Energy

info@martin-thoma.de (Martin Thoma) — Tue, 19 May 2015 19:17:00 +0200

Have you heard of GravityLight?

It is a gravity-powered lamp designed as an alternative for off-grid families who would otherwise use kersene lamps. It is basically only a 12kg weight, lifted and put on the gravity light. When the weight goes down again it pulls a cord. This cord makes an electric motor which generates electricity for LEDs. If you lift the weight 1.83m, the light lasts for about 20 minutes.

I wondered how much weight I would need to lift (assuming 100% efficiency) to power my computer for 8 hours.

According to the power supply unit, my laptop can consume up to 65 Watt. That is astonishingly low. I think my big one is at about 600-800 Watt.

\begin{align} 65 W \cdot 8 h &= \frac{65 kg \cdot m^2 \cdot 8h \cdot 60 \frac{min}{h} \cdot 60 \frac{s}{min}}{s^3}
&= 1872 \cdot 10^3 \frac{kg \cdot m^2}{s^2} \end{align}

You might also remember from your physics courses that potential energy is $E_{pot} = m \cdot g \cdot h$ where $m$ is the mass (in kg), $g = 9.80 \frac{m}{s^2}$ is the gravitational acceleration and $h$ is the height in meters.

\begin{align} m \cdot h &= \frac{E_{pot}}{g}
&= \frac{1.872 \cdot 10^6 \frac{kg \cdot m^2}{s^2}}{9.80 \frac{m}{s^2}}
&= 191.0 \cdot 10^3 kg \cdot m \end{align}

This means I would have to lift 191 000 packages one liter of milk to a height of 1 meter. Every day. Just to let my small laptop run.

Or lets view it from another angle. I think lifting about 5 packages of milk to a height of about 1.8m each hour would not be too exhausting. This would generate about $E_{pot} = 5kg \cdot 1.80m \cdot 9.80 \frac{m}{s^2} / (1h \cdot 60 \frac{min}{h} \cdot 60 \frac{s}{min}) = 0.0245 \frac{kg \cdot m^2}{s^3} = 0.0245 W$ . Lets think what you can power with 0.0245 Watt…

It is amazing about how much energy we have today.

Python Markov Chain Packages

info@martin-thoma.de (Martin Thoma) — Tue, 12 May 2015 20:45:00 +0200

Markov Chains are probabilistic processes which depend only on the previous state and not on the complete history. One common example is a very simple weather model: Either it is a rainy day (R) or a sunny day (S). On sunny days you have a probability of 0.8 that the next day will be sunny, too. On rainy days you have a probability of 0.6 that the next day will be rainy, too. As you have only two possible weather conditions, the probability that it changes from sunny do rainy is 0.2 and vice versa it is 0.4.

You can visualize this with a graph like this:

Simple Markov chain weather model

I am taking a course about markov chains this semester. Today, we’ve learned a bit how to use R (a programming language) to do very basic tasks.

R vs Python

The following will show some R code and then some Python code for the same basic tasks. As an example, I’ll use reproduction. The states are $S_1 = \{AA, AA\}$ , $S_2 = \{AA, Aa\}$ , $S_3 = \{AA, aa\}$ , $S_4=\{Aa,Aa\}$ , $S_5 = \{Aa, aa\}$ and $S_6 = \{aa, aa\}$ .

The idea is that each pair of parents give birth to two children. The parents $S_2 = \{AA, Aa\}$ can give birth to {{AA, AA}, {AA, Aa}, {Aa, Aa}}. This results in the following state transition matrix.

$\begin{pmatrix}1 & 0 & 0 & 0 & 0 & 0 \\ 1/4 & 1/2 & 0 & 1/4 & 0 & 0\\ 0 & 0 & 0 & 1 & 0 & 0\\ 1/16 & 1/4 & 1/8 & 1/4 & 1/4 & 1/16\\ 0 & 0 & 0 & 1/4 & 1/2 & 1/4\\ 0 & 0 & 0 & 0 & 0 & 1\end{pmatrix}$

The rows mean from which state you start, the colums are the states you can get to.

Now, how would you define this matrix with R?

P = matrix(c(1,0,0,0,0,0, 1/4,1/2,0,1/4,0,0, 0,0,0,1,0,0, 1/16, 1/4, 1/8, 1/4, 1/4, 1/16, 0,0,0, 1/4, 1/2, 1/4, 0,0,0,0,0,1), byrow=TRUE, nrow=6)

And this is how you do it with Python: You first need to install numpy. This is very easy with Linux (sudo apt-get install python-numpy), but I’ve heard it is not that easy with Windows systems.

import numpy as np
P = np.matrix([[1., 0., 0., 0., 0., 0.],
               [1./4, 1./2, 0., 1./4, 0., 0.],
               [0., 0., 0., 1., 0., 0.],
               [1./16, 1./4, 1./8, 1./4, 1./4, 1./16],
               [0., 0., 0., 1./4, 1./2, 1./4],
               [0., 0., 0., 0., 0., 1.]])

A common matrix operation is taking the $n$ -th power. This is how you do it with R: First, install the library “expm” by executing install.packages("expm"). Then

library("expm")
n = 5
Pn = P%^%n

and this is the Python way:

n = 5
Pn = P**n

Visualizing data is a very important tool. For example, we want to know the probabilities for the current state for the next 20 steps when you started in $S_3$ .

library("expm")
P = matrix(c(1,0,0,0,0,0,
             1/4,1/2,0,1/4,0,0,
             0,0,0,1,0,0,
             1/16, 1/4, 1/8, 1/4, 1/4, 1/16,
             0,0,0, 1/4, 1/2, 1/4,
             0,0,0,0,0,1), byrow=TRUE, nrow=6)
v = c(0,0,1,0,0,0)
for (step in 1:20) {
    matplot(t(sapply(1:20,
                     function (step) {v %*% (P %^% step)})),
                     cex=0.7,
                     ylab="")
}

This gives the following plot:

State probabilities starting in S3 after 1..20 steps (plotted with R)

The Python equivalent is

#!/usr/bin/env python

import numpy as np
from matplotlib import pyplot

P = np.matrix([[1., 0., 0., 0., 0., 0.],
               [1./4, 1./2, 0., 1./4, 0., 0.],
               [0., 0., 0., 1., 0., 0.],
               [1./16, 1./4, 1./8, 1./4, 1./4, 1./16],
               [0., 0., 0., 1./4, 1./2, 1./4],
               [0., 0., 0., 0., 0., 1.]])

v = np.matrix([[0, 0, 1, 0, 0, 0]])

# Get the data
plot_data = []
for step in range(20):
    result = v * P**step
    plot_data.append(np.array(result).flatten())

# Convert the data format
plot_data = np.array(plot_data)

# Create the plot
pyplot.figure(1)
pyplot.xlabel('Steps')
pyplot.ylabel('Probability')
lines = []
for i, shape in zip(range(6), ['x', 'h', 'H', 's', '8', 'r+']):
    line, = pyplot.plot(plot_data[:, i], shape, label="S%i" % (i+1))
    lines.append(line)
pyplot.legend(handles=lines, loc=1)
pyplot.show()

The result looks like this

State probabilities starting in S3 after 1..20 steps (plotted with Python)

I’ve played around with the matplotlib markers to make sure all points are visible.

Python Markov Packages

There seem to be quite a few Python Markov chain packages:

$ pip search markov
PyMarkovChain             - Simple markov chain implementation
autocomplete              - tiny 'autocomplete' tool using a "hidden markov model"
cobe                      - Markov chain based text generator library and chatbot
twitter_markov            - Create markov chain ("_ebooks") accounts on Twitter
markovgen                 - Another text generator based on Markov chains.
pyEMMA                    - EMMA: Emma's Markov Model Algorithms
pymc                      - Markov Chain Monte Carlo sampling toolkit.
hmmus                     - Posterior decoding with a hidden Markov model
marbl-python              - A Python implementation of the Marbl specification for normalized representations of Markov blankets in Bayesian networks.
pymdptoolbox              - Markov Decision Process (MDP) Toolbox
gibi                      - Generate random words based on Markov chains
markovgenerator           - Markov text generator
pythonic-porin            - Nanopore Data Analysis package. Provides tools for reading data,        performing event detection, segmentation, visualization, and
                            analysis using        hidden Markov models, and other tools. Designed for the UCSC Nanopore Group.
PyMarkovTextGenerator     - Random text generator base on Markov chains.
MCREPOGEN                 - Markov Chain Repository Generator
vokram                    - A toy Markov chain implementation.
MarkovEquClasses          - Algorithms for exploring Markov equivalence classes: MCMC, size counting
hmmlearn                  - Hidden Markov Models in Python with scikit-learn like API
twarkov                   - Markov generator built for generating Tweets from timelines
MCL_Markov_Cluster        - Markov Cluster algorithm implementation
pyborg                    - Markov chain bot for irc which generates replies to messages
pydodo                    - Markov chain generator
mwordgen                  - MWordGen is a Markov statistics based word generator.
Markov                    - Python library for Hidden Markov Models
markovify                 - Use Markov chains to generate random semi-plausible sentences based on an existing text.
treehmm                   - Variational Inference for tree-structured Hidden-Markov Models
PyMarkov                  - Markov Chains made easy

However, most of them are for hidden markov model training / evaluation. There seems to be no package which can visualize markov chains just by taking the state transition matrix.

There seems also not to be any package which makes it easy to classify states as transient / recurrent, get the absorption time, …

If somebody is interested in that, we could make a little project for it ☺

PyMarkovChain

Source is on github.com/TehMillhouse/PyMarkovChain. It is less than 150 lines of code and probably no functionality.

I asked the author to remove the package from PyPI (see issue #13).

pyEMMA

I’ve found the documentation and the project on PyPI. The source is on github.com/markovmodel/PyEMMA.

MarkovEquClasses

PyPI

PyMarkov

It is only about 100 lines of very simple code. It seems to be another random sentence generator. See PyPI.

Hidden Markov Models

The following might be interesting, but I didn’t take a close look at them because I was looking for “normal” markov models:

hmmus
hmmlearn
Markov
treehmm

There are also quite a few other modules which seem to generate data with markov chains.

TODO

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3097064/
https://pymc-devs.github.io/pymc/README.html
https://github.com/riccardoscalco/Pykov

Machine Learning 2

info@martin-thoma.de (Martin Thoma) — Mon, 11 May 2015 11:00:00 +0200

Dieser Artikel beschäftigt sich mit der Vorlesung „Machine Learning 2“ am KIT. Er dient als Prüfungsvorbereitung. Ich habe die Vorlesungen bei Herrn Prof. Dr. Marius Zöllner im Sommersemester 2015 gehört.
Es gibt auch einen Artikel zu Machine Learning 1

Behandelter Stoff

Einführung

Slides: 01_Einfu__hrung_MLII.pdf

Rückblick auf ML 1. MLNN steht übrigens für Multi-Layer Neural Network.

Semi-Supervised Learning

Slides: 02_Semi-supervised-learning.pdf

Überwachtes Lernen (engl. Supervised Learning)

Alle Trainingsdaten liegen mit Labels vor.

Unüberwachtes Lernen (engl. Unsupervised Learning)

Alle Trainingsdaten liegen ohne Labels vor.

Semi-Supervised Learning (SSL)

Die meisten Trainingsdaten liegen ohne Labels vor, jedoch gibt es für jede Klasse auch gelabelte Daten.

Self-Learning (Self-Training, Self-Labeling, Decision-directed learning)

Self-Training ist ein Algorithmus zum Semi-supervised Learning. Er geht wie folgt vor:

Trainiere mit gelabelten Daten.
Werte ungelabelte Daten aus.
Füge Daten, bei denen sich der Klassifizierer sicher ist, zu den Trainingsdaten hinzu.
Zurück zu Schritt 1.

Dabei sind folgende Variationen vorstellbar:

Füge alle Daten hinzu.
Füge nur Daten hinzu, bei denen sich der Klassifizierer sicher ist.
Gewichte Daten mit der Sicherheit.

Co-Training (Mit-Lernen)

Co-Training ist ein Algorithmus zum Semi-supervised Learning. Er geht wie folgt vor:

Splitte jeden Feature-Vektor auf die gleiche Art in zwei Feature-Vektoren mit disjunkten Features auf.
Trainiere zwei unterschiedliche Klassifizierer auf den beiden unterschiedlichen Feature-Mengen der gelabelten Daten.
Label mit den beiden Klassifizieren die ungelabelten Daten.
Füge ungelabelte Daten dem Trainingsdatensatz (also den gelabelten Daten) hinzu, falls die Klassifizierer für diese eine hohe Konfidenz aufweisen.
Zurück zu Schritt 2.

Dabei sind folgende Variationen vorstellbar:

Demokratisches Voting: Bei mehr als 2 Klassifizierern.
Schwellwert: Nur hinzufügen, wenn alle Klassifizierer jeweils eine Schwelle überschreiten.
Gewichtes Voting: Alle Klassifizierer zusammen müssen eine Schwelle überschreiten.

Low Density Separation

Methoden, welche Low Density separation benutze versuchen die Entscheidungsgrenze in eine Region niedriger Dichte zu legen. Ein Beispiel ist die Transductive SVM.

Weiteres:

Hier könnte ich mir gut vorstellen, dass man eine Bachelor / Master-Arbeit macht. Man könnte sich große gelabelte Datensätze suchen, einen gewissen Teil der Labels weglassen (also einige Trainingsdaten als “ungelabelt” behandeln) und die verschiedenen SSL-Methoden untersuchen. Bis zu 20% gelabelte Daten hoch wäre es interessant; also z.B. (0.5%, 1%, 2%, 3%, 5%, 10%, 15%, 20% gelabelte Daten). Mit mehr gelabelten Daten könnte man argumentieren, dass man es sich vermutlich leisten könnte auch den Rest noch zu labeln. Siehe Folie 28-31.

SSL and Active Learning

Slides: 03_Semi-supervised+Active-learning.pdf

Lagrange-Multiplikator

Lagrange-Multiplikatoren sind ein Verfahren der Optimierungstheorie. Sie werden genutzt, wenn ein Optimierungsproblem mit Nebenbedingungen vorliegt. Durch sie kann die Nebenbedingung eliminiert werden.

Active Learning

Die Lernmaschine wählt die zu lernenden Daten selbst aus.

Query Synthesis (siehe Active Learning Literature Survey)

Der Lerner kann Feature-Vektoren (Querys) de novo, also von Grund auf neu / selbst erzeugen. Er kann für diesen neuen Query ein Orakel befragen, was das Label ist.

Selective Sampling (Selektive Entnahme, siehe Selective sampling and active learning from single and multiple teachers)

Selective Sampling ist eine Methode des aktiven Lernens. Dabei wird jede Runde $t$ dem Lerner ein Feature-Vektor $x_t \in \mathbb{R}^n$ präsentiert. Der Lerner muss sich jede Runde entscheiden, ob er einen Preis bezahlt um das Label zu sehen. Der Lerner hat also zwei Ziele, die miteinander in Konflikt stehen: Er will alles richtig klassifizieren, aber zugleich die Kosten so niedrig wie möglich halten.

Pool-Based Active Learning

Pool-Based Active Learning ist eine Methode des aktiven Lernens. Dabei wird von einem Pool an ungelabelten Daten $\mathcal{U}$ ausgegangen und einem deutlich kleineren Pool $\mathcal{L}$ an gelabelten Daten. Queries werden aus $\mathcal{U}$ gezogen. Dabei wird ganz $\mathcal{U}$ evaluiert und für den hilfreichsten Feature-Vektor $x \in \mathcal{U}$ nach einem Label gefragt.

Hinge-Funktion

\[f(x) = \max(x, 0)\]

Query-by-Committee (QBC)

Es wird ein Committee $\mathcal{C}$ an Klassifikatoren trainiert, welches gemeinsam (z.B. durch majority vote) eine Klassifikation trifft. Allgemeiner Ansatz:

Trainiere eine Menge $\mathcal{C}$ an Klassifikatoren
Wähle neue Daten, wenn die Hypothesen Wiedersprüchlich sind

Selektive Entnahme:

Beobachte neue Instanz $x$ und werte diese mit $\mathcal{C}$ aus
Frage das Label ab, falls es einen Wiederspruch in den Hypothesen von $\mathcal{C}$ für $x$ gibt.
Neu trainiren, zurück zu 1

Pool-based Active Learning:

Messung des Wiederspruchs der Hypothesen für alle Instanzen $x$
Ranking (z.B. Entropie)
Abfrage der Labels für die $k$ widersprüchlichsten Instanzen
Neu trainiren, zurück zu 1

Weiteres:

Ausreißerproblem: Ausreißer sollten im QBC nicht genommen werden. Dazu könnte die Dichte im Datenraum gemessen werden.

Reinforcement Learning

Slides: 04_Reinforcement_Learning_II.pdf

Siehe auch:

Neuronale Netze
Machine Learning 1
Cat vs. Mouse code
Berkeley
- CS188 Intro to AI: Project 3: Reinforcement Learning
- Dan Klein, Pieter Abbeel: Lecture 10: Reinforcement Learning on YouTube. University of California, Berkeley. This expalins TD-learning.

Markov Decision Process (MDP)

Ein Markovscher Entscheidungsprozess ist ein 5-Tupel $(S, A, T, r, p_0)$, wobei

$S$ eine endliche Zustandsmenge,
$A$ eine endliche Menge von Aktionen,
$T_a(s, s') = T(s_{t+1}=s'|s_t = s, a_t = a)$ die Wahrscheinlichkeit zu einem beliebigen Zeitpunkt von Zustand $s$ mit der Aktion $a$ in den Zustand $a'$ zu kommen (engl. Transition),
$r_a(s, s')$ ist die Belohnung (Reward), die man direkt erhält wenn man erhält wenn man von Zustand $s$ mit Aktion $a$ in Zustand $s'$ kommt,
$p_0$ ist die Startverteilung auf die Zustände $S$

Partially observable Markov decision process (POMDP)

Ein partially observable Markov decision process ist ein 7-tupel $S, A, T, R, \Omega, O, \gamma$, wobei

$S$ die Zustandsmenge,
$A$ die Aktionsmenge,
$T: S \times A \times S \rightarrow \mathbb{R}$ die probabilisitische Zustandsübergangsfunktion (transition function) ist,
$R: S \times A \rightarrow \mathbb{R}$ die Reward-Funktion,
$\Omega$ die Menge der möglichen Beobachtungen,
$O$ die Wahrscheinlichkeit der Beobachtungen, gegeben ein Zustand und eine Aktion und
$\gamma \in [0, 1]$ der Diskontierungsfaktor

ist.

Options

Eine Option ist wohl-definiertes Verhalten, welches im hierarchischen RL eingesetzt werden kann. Es ist ein Baustein für komplexe Pläne. Options werden in Semi-MDPs eingesetzt und ersetzen dort die Aktionen.

Hierarchien Abstrakter Maschinen (HAM)

Ein MDP wird mit Maschinen $\{M_i\}$ kombiniert. Jede Maschine repräsentiert einen Teil der Policy. Jede Maschine verwendet eigene Zustände $m_t^i$ und globale Zustände $s_t$. Maschinen werden durch Zustandsautomaten abgebildet.

MaxQ-Dekomposition (siehe [Die00])

Das zu lösende MDP $M$ wird als Menge von Unteraufgaben $\{M_0, \dots, M_n\}$ interpretiert. Dabei ist $M_0$ das Haupt-MDP. TODO.

Folie 35:

NODO: Was heißt hier “mit festen Knoten”?

Dynamische Bayessche Netze

Slides: 05_DynamischeBayesscheNetze.pdf

Multiplikationssatz

Seien $A, B, X_i$ Ereignisse. Dann gilt: \[P(X_1, \dots, X_n) = P(X_1) \cdot \prod_{k=2}^n P(X_k | X_{k-1}, \dots, X_1)\] und insbesondere \[P(A\cap B) = P(A, B) = P(A\mid B) \cdot P(B)\]

Gesetz der totalen Wahrscheinlichkeit

Seien $A_1, \dots, A_n$ paarweise disjunkte Ereignisse mit $A = \sum_{i=1}^n A_i$. Dann gilt für jedes beliebige Ereignis $B$: \[P(B) = \sum_{i=1}^n P(B | A_i) \cdot P(A_i) = P(A_i, B)\]

Satz von Bayes

Seinen $A, B$ Ereignisse mit $P(B) > 0$. Dann gilt \[P(A|B) = \frac{P(B|A) \cdot P(A)}{P(B)}\] Hierbei heißt $P(A|B)$ die a posteriori Wahrscheinlichkeit, $P(B|A)$ die likelihood, $P(A)$ die a priori Verteilung über $A$ und $P(B)$ die a priori Verteilung über $B$.

Bayessches Netz (Siehe Lecture 13: Bayes Nets)

Ein Bayessches Netz ist ein DAG, bei dem die Knoten Zufallsvariablen und die Kanten bedingte Abhängigkeiten beschreiben. Bayessche Netze sind zur Modellierung kausaler Zusammenhänge geeignet.

Markov Random Field

Siehe ML 1

Dynamisches Bayessches Netz

Dynamische Bayessche Netze sind Bayessche Netze zur Beschreibung dynamischer Prozesse.

Markov Blanket

Sei $G=(V,E)$ ein DAG zu einem Bayesschen Netz und $v_S \in V$. Dann ist der Markov Blanket die folgende Knotenmenge $B \subseteq V \setminus \{v_S\}$:

Die Elternknoten von $v_S$ sind in $B$.
Die Kindknoten $K = \{v_{K_1}, \dots, v_{K_n}\}$ sind in $B$
Die Elternknoten von $K$, ausgenommen von $v_S$, sind in $B$

Diese Knotenmenge macht $v_S$ unabhängig von anderen Knoten.

Naive Bayes Spam Filter

Ein naiver Bayes Spamfilter nutzt häufig Bag-of-Words Features. Man berechnet die Wahrscheinlichkeit, dass eine gegebene E-Mail Spam ist. Dazu geht man davon aus, dass die Wörter in einer E-Mail unabhängig von einander sind und nutzt den Satz von Bayes. Siehe Bayes-Klassifikator für eine detailiertere Beschreibung.

Bayes Filter

Ein Bayes Filter ist eine Familie von Zufallsvariablen. Das könnte z.B. die $(x,y,z)$ Position eines GPS-Sensors sein. Diese Position ist verrauscht. Nun gibt es drei mögliche Anfragen:

Filtern: Es liegen Messungen $Z_0, \dots, Z_t$ vor, sage die aktuelle Position $X_t$ vorher. Also filtere das Rauschen aus $Z_t$ unter berücksichtigung, dass wir uns noch nicht teleportieren können: \[P(X_t | Z_t, \dots, Z_0)\]
Prädizieren: Es liegen Messungen $Z_0, \dots, Z_t$ vor, sage die Position $X_{t+k}$ vorher: \[P(X_{t+k} | Z_t, \dots, Z_0)\]
Glätten: Es liegen Messungen $Z_0, \dots, Z_t$ vor, sage die Position $P(X_{t-k} | Z_t, \dots, Z_0)$ vorher.

Beispiele für Bayes-Filter sind

Kalman-Filter
HMM
Partikel Filter

Naiver Bayes'scher Spam Filter

Ein probabilistischer Klassifikator welcher die Unabhängigkeit der Features vorraussetzt wird naiv genannt.

Der naive bayessche Spam Filter nutzt Bayes Theorem um die Wahrscheinlichkeit zu berechnen, dass eine E-Mail Spam ist.

Kalman-Filter (siehe Python-Implementierung)

Der Kalman-Filter ist ein Bayes-Filter. Er wird z.B. zum Schätzen einer Fahrzeugtrajektorie eingesetzt. Der Kalman-Filter besteht aus zwei Schritten:

Predict the next step of the system given the previous measurements.
Update the estimate of the current state given the measurement of this time step.

Expectation Maximizaion Algorithm (EM-Algorithmus)

Der EM-Algorithmus kann als ein Clusteringalgorithmus mit weicher Clusterzugehörigkeit gesehen werden. Er findet die Parameter für gegebene Verteilungen (üblicherweise multivariate Normalverteilungen). Er löst das Henne-Ei Problem

Wenn man weiß wie genau die Wahrscheinlichkeitsverteilungen der Cluster parametrisiert sind ist es leicht die Daten den Clustern zuzuordnen.
Wenn man die Daten einem Cluster zuordnen kann, dann ist es leicht die Parameter der Wahrscheinlichkeitsverteilung zu schätzen.

Wenn man sowohl Clusterzugehörigkeit als auch die Parameter der Verteilung schätzen muss ist es schwer. Man kann "zufällig" die initialen Parameter wählen, dann die Zuordnung machen. Der EM-Algorithmus iteriert also:

Expectation: Weise alle Datenpunkte ihrem Cluster zu.
Maximization: Berechne die Parameter der Cluster neu.

Siehe Mixture Models 1: the EM algorithm

Typische Fragestellungen:

Gegeben ist die Struktur eines Bayesschen Netzes: Wie lautet die Verteilung?
- Dies wird üblicherweise mit dem EM-Algorithmus gelöst.

Anwendungsfälle:

Automatische Diagnose, gegeben die Symptome (Bayessches Netz)
Fahrzeugverfolgung: Vorhersage von Routen, welche die Fahrzeuge nehmen werden (Dynamisches Bayessches Netz)

Anmerkungen: Die Folien sind hier sehr gut! Insbesondere Folie 14-23 sollte man sich ansehen.

Es scheint folgende Beziehung zu gelten: HMMs, Kalman-Filter, Extended Kalman-Filter, Partikel Filter sind Beispiele für Bayes-Filter. Bayes-Filter sind Beispiele für dynamische Bayessche Netze.

Siehe auch:

Udacity: Artificial Intelligence for Robotics - good content for Kalman Filters

Tracking Robots
Probabilistic Graphical Models

Probablistisch Relationale Modelle

Slides: 06_Probablistisch_Relationale_Modelle.pdf

Siehe auch:

C. Howard and M. Stumptner and others. Model Construction Algorithms for Object-Oriented Probabilistic Relational Models. FLAIRS Conference, 2006.
C. Howard and M. Stumptner. Situation assessments using object oriented probabilistic relational models, in 8th International Conference on Information Fusion, 2005, vol.2. doi: 10.1109/ICIF.2005.1592031

Objektorientierte Probablistisch Relationales Modelle (OPRM)

Ein OPRM besteht nach [Schu15] aus

Eine Klassenmenge $\mathbf{C} = \{C_1, \dots, C_n\}$,
einer partiellen Ordnung über C, welche die Klassenhierarchie definiert,
einer Menge einfacher, nicht probabilisitscher Attribute $\Lambda_C = \{\lambda_1, \dots, \lambda_n \forall C \in \mathbf{C}\}$,
einer Menge beschreibender Attribute $\Delta_C = \{\delta_1, \dots, \delta_n\} \forall C \in \mathbf{C}$,
einer Menge komplexer Attribute $\Phi_C = \{\phi_1, \dots, \phi_n\} \forall C \in \mathbf{C}$. Die komplexen Attribute beschreiben funktionale Beziehungen zwischen Klassen.

Dieses Modell wurde in libDAI umgesetzt.

Gaussche Prozesse

Slides: 07_Gaussche_Prozesse.pdf

I suggest reading the first two chapters of the online book gaussianprocess.org before starting to read the slides.

See also:

Function Approximation
The Talking Machines: OpenAI and Gaussian Processes

Lineare Regression: Die lineare Regression ist ein Modell zur approximation von Datenpunkten $(x, y) \in \mathbb{R}^n \times \mathbb{R}$ durch eine lineare Funktion, d.h. einer Funktion der Form $f(x) = x^T \cdot w$. Dabei ist $w \in \mathbb{R}^n$.

Wenn man als Optimierungskriterium den quadratischen Abstand \[E(f, data) = \sum_{(x,y) \in data} (f(x) - y)^2\] nimmt, dann ist eine optimale Lösung durch \[w = (X^T X)^{-1} X^T y\] gegeben.

Siehe auch: Proof of when is $A=X^T X$ invertible? sowie Does a transformation + linear regression give the same regression as fitting a quadratic function?
Affine Regression: Die affine Regression ist ein Modell zur approximation von Datenpunkten $(x, y) \in \mathbb{R}^n \times \mathbb{R}$ durch eine affine Funktion, d.h. einer Funktion der Form $f(x) = x^T \cdot w + b$. Dabei ist $w \in \mathbb{R}^n, b \in \mathbb{R}$. Um das Problem auf ein lineares zu reduzieren kann man den Feature-Vektor $x$ durch ein konstantes Feature $x_0 = 1$ erweitern.
Korrelationskoeffizient: Der Korrelationskoeffizient $\kappa(X, Y) \in [-1, 1]$ ist ein Maß für den linearen Zusammenhang zwischen zwei Zufallsvariablen $X, Y$. Er ist definiert als \[\kappa(X, Y) := \frac{Cov(X, Y)}{\sigma(X) \cdot \sigma(Y)}\]
Gausscher Prozess (Kriging, Machine learning - Introduction to Gaussian processes by Nando De Freitas): Gaussche Prozesse approximieren eine Funktion dadurch, dass sie an jedem Punkt eine Normalverteilung (Gauss-Verteilung) annehmen.

Siehe Gaussian process regression.

Deep Learning

Slides: 08_DeepLearning.pdf

Siehe auch:

Neuronale Netze Vorlesung
Udacity: Neural Networks for Machine Learning by Hinton.

Deep Belief Netz (DBN)

Ein Deep Belief Netz ist ein gerichtetes, azyklisches, probabilistisches graphisches Modell.

Restricted Boltzmann Machine (RBM)

Eine RBM ist ein neuronales Netz mit nur einem Hidden Layer. Es ist gleichzeitig ein Spezialfall von MRFs. Es werden keine Verbindungen zwischen den Hidden Units erlaubt (daher das "restricted" - Quelle: Hinton, 2015).

Siehe A Practical Guide to Training Restricted Boltzmann Machines von Hinton, 2010.

Contrastive Divergence (CD, siehe YouTube Video von Hugo Larochelle)

Contrastive Divergence ist ein Trainingsalgorithmus für RBMs. Ein Hyperparameter ist $k \in \mathbb{N}$. Er geht wie folgt vor:

Lege den Trainingsvektor $x^{(t)}$ an die Eingabeknoten an.
Berechne die Wahrscheinlichkeit für jede Hidden Unit, dass diese gleich 1 ist. Setze sie mit dieser Wahrscheinlichkeit gleich 1.
Berechne die Wahrscheinlichkeit für jeden Eingabeknoten, dass dieser gleich 1 ist. Setze ihn mit dieser Wahrscheinlichkeit gleich 1.
Gehe zu Schritt 2. Wiederhole dies für $k$ Schritte (dies wird auch Gibbs-Sampling genannt). Das, was nach dem $k$-fachem Gibbs-Sampling in der Eingabeschicht steht wird auch "negative sample $\tilde x$" genannt.
Update der Parameter: \[\begin{align} W &\leftarrow W + \alpha (h(x^{(t)}) {x^{(t)}}^T - h(\tilde x) {\tilde x}^T)\\ b &\leftarrow b + \alpha (h(x^{(t)}) - h(\tilde x))\\ c &\leftarrow c + \alpha (x^{(t)} - \tilde x) \end{align} \] wobei $\alpha \in (0, 1) $ die Lernrate ist, $b \in \mathbb{R}^n_h$ der Bias-Vektor der Hidden Units und $c \in \mathbb{R}^{n_v}$ der Bias-Vektor der Eingabeknoten ist. $h$ ist eine Zufallsvariable, welche der Hidden Layer ist. Diese sind abhängig von der Eingabeschicht.

In der Praxis funktioniert es schon mit $k=1$ für Pre-Training. Wenn $k$ groß ist konvergiert $\tilde x$ gegen den wahren Modellwert. Das wäre dann eine Monte-Carlo Estimation.

Contrastive Wake-Sleep Algorithm (siehe The wake-sleep algorithm von Hinton - Lecture 13d aus "Neural Networks for Machine Learning")

Der Wake-Sleep Algorithmus ist ein Trainingsalgorithmus für gerichtete graphische Modelle wie Sigmoid Belief Networks. Er ist nicht für RBMs. Man hat im Grunde zwei Netzwerke mit der gleichen Topologie, jedoch ist die Richtung vertauscht: Das eine Netz stellt die Hypothese aus den Daten auf, das andere Netz geniert neue Daten aus einer gegebenen Hypothese. In der wake phase wird die Eingabe genutzt um die Hypothese zu erzeugen. In dieser Phase werden die Gewichte für das Generative Modell trainiert. Dieses soll die Aktivierung der vorhergehenden Schicht rekonstruieren. In der sleep phase wird das generative Modell genutzt um aus dem Modell samples zu erzeugen. Dann trainiert man die Gewichte des erkennenden Netzes (also vergleichbar mit der wake phase, nur anders rum).
Siehe A Fast Learning Algorithm for Deep Belief Nets

Probleme von Tiefen Netzen und wie man sie lösen kann:

Lange Trainingsdauer: GPUs / mehr Rechenpower / weniger Parameter durch Parameter sharing, z.B. in CNNs / TDNNs
Extrem viele gelabelte Trainingsdaten werden benötigt: Internet (z.B. Wikipedia, Soziale Netzwerke, Amazon Mechanical Turk) reduziert dieses Problem; Nutzen ungelabelter Daten durch SSL in Auto-Encodern
Lokale Minima
Overfitting: Regularis

Siehe auch

MNIST Demo (Flash): Neuronales Netz welches Ziffern generiert
Geoffry Hinton: Deep Learning on YouTube, 2015. 43 minutes. (Topics: RBMs)

Convolutional Neural Networks

Slides: 09_ConvolutionalNeuralNetworks.pdf

Siehe auch: Neuronale Netze Vorlesung

Convolutional Neural Networks (CNNs)

CNNs sind neuronale Netze welche weight sharing einsetzen. Sie setzen eine diskrete Faltung um. Ein CNN muss mindestens einen Convolutional Layer haben. Dieser hat folgende Parameter:

Padding: None, Zero, Copy
Stride: $s \in \mathbb{N}_{> 0}$
Filter Size: $(x,y) \in \mathbb{N}^2$
Number of filters: How many filters should get learned?

Feature Map

Nach einem Convolutional Layer hat man die Ausgabe der Filter, welche auf die Eingabe angewandt wurden. Diese nennt man Feature Map. Für jeden Filter bekommt man eine Feature Map. Die Feature Maps sind wiederum Eingaben für die nächsten Schichten.

Pooling Layer

Ein pooling layer ist eine Schicht in einem CNN, welche Features zusammenfasst. Pooling Schichten haben folgende Parameter:

Größe: Typischerweise $3 \times 3$
Stride $s \in \mathbb{N}$: Typischerweise gleich der Größe des Pooling-Bereichs (also 3).
Art: max, mean

Typischerweise reduziert sie die Anzahl der Features, da typischerweise ein $s > 1$ gewählt wird.

Spiking Neural Nets

Slides: 10_SpikingNeuralNets.pdf

Spiking Neural Networks: Gepulste neuronale Netze versuchen natürliche neuronen realistisch abzubilden. Das Hodgkin-Huxley Neuronenmodell wurde bereits 1952 vorgestellt.
Hodgkin-Huxley Neuronenmodell: Das Hodgkin-Huxley Neuronenmodell modelliert die elektrochemischen Vorgänge innerhalb eines Neurons mit elektrischen Baugliedern. Dies resultiert in Differenzialgleichungen mit 4 Variablen (Kapazität der Membran, Widerstände der Ionenkanäle, Gleichgewichtspotentiale, Öffnung der Ionenkanäle). Das Modell ist realistisch, aber sehr komplex.
LIF Neuronenmodell (Leaky integrate and Fire): Das LIF Neuronenmodell modelliert ein Neuron durch eine gewöhnliche Differentialgleichung erster Ordnung.
SRM Neuronenmodell (Spike Response Model): Das SRM Neuronenmodell modelliert die Refraktionszeit. Das ist die Zeit, in der kein neues Aktionspotential aufgebaut werden kann. Das SRM ist ein rein phänomenologisches Modell, welches trotz der Einfachheit allgemeiner ist als das LIF-Modell.

Evaluation

Slides: 11_Evaluation.pdf

Für Klassifikation:

Konfusionsmatrix: Eine Konfusionsmatrix ist eine Tabelle, in welcher die Spalten angeben, welche Hypothese gemacht wurde (Testentscheid) und die Zeilen den wahren Wert angeben. So kann für beliebig viele Klassen gezeigt werden, wie gut der Klassifikator ist und welche Art der Verwechslung er macht.
Klassifikationsfehler: $\text{Klassifikationsfehler} = \frac{\text{Fehlerhafte Hypothesen}}{\text{Anzahl aller Beispiele}} \in [0, 1]$
Klassifikationsgüte: Klassifikationsgüte = 1 - Klassifikationsfehler
False Alarm Rate (FA, Falsch Positiv Rate, FPR): Es sei FP die Anzahl der False Positive Testdaten, also der Testdaten für welche Positive vorhergesagt wurde, die aber negative sind. Weiter sei TN die Anzahl der True Negatives, also der Testdaten, für welche korrekterweise negative vorhergesagt wurde. Dann ist die FPR definiert als \[\text{FPR} := \frac{FP}{FP + TN} \in [0, 1]\] Die FPR gibt also den Anteil an, wie viele der tatsächlich negativen fälschlicherweise als positiv erkannt wurden.
Miss-Rate (MR, Falsch Negativ Rate, FNR): \[FNR := \frac{FN}{TP + FN} \in [0, 1]\]
Recall (True Positive Rate, TPR, Sensitivität): \[TPR = \frac{TP}{TP + FN} = 1 - FNR \in [0, 1]\] Der Recall gibt den Anteil der erkannten positiven aus allen positiven an. Sensitivität ist ein in der Medizin üblicher Begriff.
Precision (Genauigkeit): \[Precision = \frac{TP}{TP + FP} \in [0, 1]\] Die Precision gibt den Anteil der real positiven aus den als positiv erkannten an.
ROC-Graph (Receiver-Operator Curve): Der ROC-Graph gibt für einen Klassifikator, bei dem man einen Parameter einstellen kann, den Fehler an. Die $x$-Achse ist dabei die FPR, die $y$-Achse die TPR.
Spezifität: Der Begriff der Spezifität ist in der Medizin üblich und ist definiert durch \[Spezifität = \frac{TN}{TN + FP} = 1 - FPR\] Es ist eine Art recall für die negative Klasse. Im Beispiel eines medizinischen Tests wäre das der Anteil der Gesunden, bei denen tatsächlich auch die Diagnose "Gesund" gestellt wurde.
PRC-Graph (Precision-Recall-Graph): Die $x$-Achse ist Recall, die $y$-Achse ist Precision.
F-Maß: \[F_\alpha = \frac{precision \cdot recall}{\alpha^2 \cdot precision + recall}\]

Alternative:

Aufstellen einer Kostenfunktion und optimieren nach Kosten.
Plotten der Anzahl der Trainingsdaten ($x$-Achse) und des Fehlers ($y$-Achse). Die Kurven sollten der Test-Fehler sowie der Trainingsfehler sein. Damit lässt sich abschätzen, ob mehr Trainingsdaten ohne eine Veränderung des Modells hilfreich sind.

Für Regression

Mittlerer Quadratischer Fehler (MSE, Mean Squared Error): \[E(f, data) = \frac{1}{|data|} \sum_{(x, y) \in data} (f(x) - y)^2\]
Relativer Quadratischer Fehler: \[E(f, data) = \frac{\sum_{(x, y) \in data} (f(x) - y)^2}{\sum_{(x,y) \in data} (y - \mu)^2}\]
Mittlerer Absoluter Fehler: \[E(f, data) = \frac{1}{|data|} \sum_{(x, y) \in data} |f(x) - y|\]

Siehe auch

Beurteilung eines binären Klassifikators
False positives and false negatives
Matt Zeiler: Visualizing and Understanding Deep Neural Networks on YouTube, 2015. 48 minutes.

Prüfungsfragen

Was versteht man unter einer "Transductive SVM"?
→ Eine Transductive SVM ist eine SVM welche neben gelabelten Daten auch noch ungelabelte benutzt. Sie versucht die Trennebene durch eine Region geringer Dichte zu legen.
Wie lautet die Optimierungsformel der transductive SVM?
→ \[\text{minimize}_{w, b, y^*} \frac{1}{2} \|w\|^2\] unter den Nebenbedingungen \[\forall i \in 1, \dots, n: y_i (w \cdot x_i - b) \geq 1\] und \[\forall j \in 1, \dots, k: y_j^* (w \cdot x_j^* -b) \geq 1\text{ with }y_j^* \in \{-1, 1\}\] Dabei sind $D^* = \{x_i^* | i = 1, \dots, k\}$ ungelabelte Daten.
Was macht man im Reinforcement Learning, wenn Aktionen länger dauern?
→ Options verwenden (TODO: Wie ändert sich die Value Iteration Formel nun bzgl. der Zeit?)
Warum heißen POMDPs "Partially Observable"?
→ Weil der Agent zwar Feedback über die Umgebung bekommt, aber nicht direkt erfährt in welchem Zustand er ist. Siehe Definition.
Welche Active Learning Techniken gibt es?
→ Query / Selective / Pool-based (vgl. Query-by-Committee)
Wie nennt man ein instanziiertes OPRM?
→ TODO (Skelett?)
Wie funktioniert aktives Lernen bei SVMs?
→ Bei SVMs gibt es die Dualität zwischen dem Feature-Space und dem Hypothesenraum. In dem Feature-Space stellen die Achsen $x_i$ die Features dar, Trainingsdaten Punkte sind und die SVM durch die Trennebene visualisiert wird. Im Hypothesenraum sind die Achsen $w_i$ zusammen der Normalenvektor der SVM, die verschiedenen Trennebenen der SVMs sind hier Punkte. Die Daten geben Bedingungen an die SVM vor, welche in diesem Raum als Hyperebenen dargestellt werden können. Der Margin ist in diesem Raum ein Kreis, der die Bedingungs-Hyperebenen berührt. Beim aktiven lernen versucht man den Version-Space im Inneren der Bedungungs-Hyperebenen so schnell zu verkleinern wie möglich.
Was versteht man unter Transduktivem Lernen?
→ Unter Transduktiver Inferenz versteht man das Schließen von Trainingsbeispielen direkt auf auf spezifische Testfälle.
Wie nennt man die Wahrscheinlichkeit des aktiellen Zustands in POMDPs?
→ Belief.

Material und Links

Literatur

[Schu15] J. Schulz. Erkennung von Interaktionen zwischen Verkehrsteilnehmern zur Verhaltensprädiktion. Masterarbeit am FZI. Karlsruhe, 2015. Man kann Florian Kuhnt um Zugang dazu fragen.
[Die00] T. Dietterich. Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition. Journal of Artificial Intelligence Research, 2000.

Übungsbetrieb

Es gibt keine Übungsblätter, keine Übungen, keine Tutorien und keine Bonuspunkte.

Kontakt

goettl@fzi.de: Sonja Göttl (Sekretariat, zum Anmelden zur mündlichen Prüfung)

Termine und Klausurablauf

Neuronale Netze - Klausur

info@martin-thoma.de (Martin Thoma) — Mon, 27 Apr 2015 21:15:00 +0200

Dieser Artikel beschäftigt sich mit der Vorlesung „Neuronale Netze“ am KIT. Er dient als Prüfungsvorbereitung. Ich habe die Vorlesungen bei Herrn Prof. Dr. Alexander Waibel im Sommersemester 2015 gehört. Der Artikel wird bis zur mündlichen Prüfung laufend erweitert.

Behandelter Stoff

Vorlesung

Datum	Kapitel	Inhalt
15.04.2015	Einleitung	-
21.04.2015	LVQ and related Techiques	k-Means, OLVQ1, kompetitives Lernen, Mode Seeker, PCA
22.04.2015	-	Übung
28.04.2015	Perceptron	-
12.05.2015	Auto-Encoder	Denoising- und Sparse Autoencoder, Bottleneck-FeaturesKullback-Leibler-Divergenz; Kettenregel
13.05.2015	Deep Learning	Momentum, Rprop, Newbob, L1/L2-Regularisierung ($\|w\|$, $w^2$), weight decay
19.05.2015	Übung	-
20.05.2015	Übung	-
26.05.2015	SOM	Hebbian Learning, "VQ" mit SOM
09.06.2015	Effizientes Lernen	Paralleles Lernen; Quickprop; Alternative Fehlerfunktion (cross entropy, CFM); weight elimination / regularization
15.07.2015	Summary	When to use which objective function (cross entropy, MSE); Backpropagation; Weight initialization; Regularization (L2 weight decay, dropout); Time Delay NN; Recurrent Networks; Applications (Speech Recognition, Computer Vision)

NN01-Intro.pdf

Human Brain vs. Computer (Processing/Processors, Accuracy, Speed, Hardware, Design)
Aufbau eines biologischen Neurons (vgl. Wikipedia)

NN02-Classification.pdf

Rosenblatt-Perceptron which realizes logical or

McCullch-Pitts Neuron (weights, bias, activation function is step function)
Rosenblatt Perceptron Algorithmus
Backpropagation
Curse of Dimensionality
Parzen Window
Features: Nominal, Ordinal, Intervallskaliert, Verhältnisskaliert

Bayes-Rule (Source: wikipedia): Given events $A_1$, $A_2$ and $B$, Bayes' rule states that the conditional odds of $A_1:A_2$ given $B$ are equal to the marginal odds of $A_1:A_2$ multiplied by the Bayes factor or likelihood ratio $\Lambda$: \[O(A_1:A_2|B) = \Lambda(A_1:A_2|B) \cdot O(A_1:A_2) ,\] where \[\Lambda(A_1:A_2|B) = \frac{P(B|A_1)}{P(B|A_2)}.\]
Parametrischer Klassifizierer: Ein Klassifizierer heißt parametrisch, wenn er eine Wahrscheinlichkeitsverteilungsannahme macht.
Naiver Bayes-Klassifikator: Ein Klassifizierer heißt naiver Bayes-Klassifikator, wenn er den Satz von Bayes unter der naiven Annahme der Unabhängigkeit der Features benutzt.
Normalverteilung: Eine stetige Zufallsvariable $X$ mit der Wahrscheinlichkeitsdichte $f\colon\mathbb{R}\to\mathbb{R}$, gegeben durch
$f(x) = \frac {1}{\sigma\sqrt{2\pi}} e^{-\frac {1}{2} \left(\frac{x-\mu}{\sigma}\right)^2}$
heißt $\mathcal N\left(\mu, \sigma^2\right)$-verteilt, normalverteilt mit den Erwartungswert $\mu$ und Varianz $\sigma^2$.
Multivariate Normalverteilung (vgl. Wikipedia): Eine $p$-dimensionale reelle Zufallsvariable $X$ ist normalverteilt mit Erwartungswertvektor $\mu$ und (positiv definiter) Kovarianzmatrix $\Sigma$, wenn sie eine Dichtefunktion der Form \[f_X(x)=\frac{1}{ \sqrt{(2\pi)^p \det(\Sigma)} } \exp \left( -\frac{1}{2}(x-\mu)^{T}\Sigma^{-1}(x-\mu) \right)\] besitzt. Man schreibt \[X\sim \mathcal N_p(\mu, \Sigma).\]
Gauß'scher Klassifizierer: Ein (naiver) Bayes-Klassifikator, welcher von normalverteilten Daten ausgeht heißt Gauß'scher Klassifizierer.
Principal Component Analysis (PCA, Hauptkomponentenanalyse - vgl. Wikipedia): Die Hauptkomponentenanalyse ist ein Verfahren zur Dimensionalitätsreduktion von ungelabelten Daten im $\mathbb{R}^n$. Sie projeziert die Daten auf diejenige Hyperebene im $\mathbb{R}^d$, die den durch die Projektion stattfindenden Datenverlust minimal hält. Dabei ist $d \in 1, \dots, n$ beliebig wählbar. Die Transformation der Daten $X$ findet durch eine Matrixmultiplikation $Y = P \cdot X$ statt. Die Matrix $P$ besteht aus den ersten $d$ Eigenvektoren der Kovarianzmatrix der Features $X$: $P = (v_1, \dots, v_d)$ mit $\lambda_j v_j = C_X v_j$ für $j=1,\dots,d$ Außerdem gilt: $C_X = \frac{1}{n-1} X X^T$

V04: Perceptron

Slide name: V04_2015-04-28_Perceptron.pdf

McCulloch–Pitts (MCP) Neuron: Ein MLP-Neuron is ein Algorithmus zur binären Klassifizierung. Er hat $m+1$, mit $m \in \mathbb{N}_{> 0}$ inputs $x_i \in \{0, 1\}$. Davon ist der erste (nullte) Konstant gleich Eins und wird Bias genannt. Jeder Input wird mit eingem Gewicht $w_i \in \mathbb{R}$ multipliziert, alle gewichteten Inputs werden addiert und schließlich wird die Stufenfunktion $\varphi(x) = \begin{cases}1 &\text{falls } x > 0\\0 &\text{sonst} \end{cases}$ angewendet.
Rosenblatt-Perzeptron: Wie das McCulloch–Pitts (MCP) Neuron, nur ist $x_i \in \mathbb{R}$ und ein Lernalgorithmus ist gegeben. Dieser addiert den $\lambda \in (0, 1)$ gewichteten, fehlklassifizierten Vektor auf die Gewichte $w_i$. $\lambda$ heißt die Lernrate.
Pocket Perceptron Algorithm: Ein Lernalgorithmus für ein Rosenblatt-Perzeptron. Dieser konvergiert zu Gewichten, welche die wenigsten Beispiele falsch klassifiziert.
Sigmoid-Funktion: $\varphi(x) = \frac{1}{1+e^{-x}}$
Softmax-Funktion: $\varphi(a_i) = \frac{e^{a_i}}{\sum_{k} e^{a_k}}$ wobei $a_i$ die Aktivierung des $i$-ten Neurons der selben Schicht ist.
Perzeptron / Logistic Neuron: MSE + Sigmoid activation function

Fakten:

Das Rosenblatt-Perzeptron findet eine lineare Trenngrenze, wenn sie existiert.
Probleme vom Rosenblatt-Perzeptron:
- XOR
- Nicht-linear trennbare Daten
- Nicht-trennbare Daten
- Wahl der Lernrate und der Startgewichte
Aufbau eines biologischen Neurons (Axon, Dendriten, Zellkörper, Ranviersche Schnürringe, Synapsen)
Glia-Zellen

Fragen:

Folie 6: Ist der Input nicht in [0, 1]?

V05: Features

Slide name: V05_2015-04-29_Features.pdf

Rectified Linear Unit (ReLU)

$\varphi(x) = \max(0, x)$

Softplus

$\varphi(x) = \log(1 + e^x)$

Feed Forward Neural Network

A Feed Forward Neural Network is a learning algorithm which takes a fixed-size input feature vector, applies varous matrix multiplications and point-wise non-linear functions to obtain a fixed-size output vector.

Multilayer Perceptron

A Multilayer Perceptron is a special type of Feed Forward Neural Network. It consists of fully connected layers only.

Metrik (siehe Wikipedia)

Sei $X$ eine Menge und $d:X \times X \rightarrow \mathbb{R}$ eine Abbildung. $d$ heißt Metrik auf $X$, wenn gilt:

$d(x, y) = 0 \geq x=y \;\;\; \forall x, y \in X$
$d(x,y)=d(y,x=$
$d(x,y) \leq d(x,z) + d(z,y)$

Jaccard-Metrik (siehe Wikipedia)

Es seien $A, B$ Mengen und $J(A, B) := \frac{|A \cup B| - |A \cap B|}{|A \cup B|}$. Dann heißt $J$ die Jaccard-Metrik.

Levenshtein-Distanz (siehe Wikipedia)

Es seien $a, b$ Zeichenketten, $|a|$ die Länge der Zeichenkette $a$ und $\delta_{a_i \neq b_j}$ genau dann 1, wenn das $i$-te Zeichen von $a$ und das $j$-te Zeichen von $b$ sich unterscheiden. Dann heißt $d_L(a, b)$ die Levenshtein-Distanz: \[d_L(a,b) := lev_{a,b}(|a|, |b|)\] \[\text{lev}_{a,b}(i, j) = \begin{cases}\max(i,j) &\text{falls} \min(i,j) = 0,\\ \min \begin{cases}\text{lev}_{a,b}(i-1,j)+1\\ \text{lev}_{a,b}(i,j-1)+1\\ \text{lev}_{a,b}(i-1,j-1)+\delta_{(a_i \neq b_j)}\\\end{cases} &\text{sonst}\end{cases}\]

Fragen:

Welche Feed Forward Neuronalen Netze existieren, die keine Multilayer Perceptronen sind?
Welche Skalentypen gibt es für Merkmale (Features)?
- Nominale Merkmale: Nur Gleichheit kann überprüft werden
- Ordinale Merkmale: Es existiert eine “kleiner gleich”-Relation
- Intervallskalierte Merkmale:
  - Die Differenz der Merkmale hat eine Semantik
  - Es existiert jedoch kein “wirklicher” Nullpunkt
- Verhältnisskalierte Merkmale: Wie Intervallskaliert, aber mit absolutem Nullpunkt.

V06: Backpropagation

Slide name: V06_2015-05-05_Backpropagation.pdf

Kreuzentropie Fehlerfunktion (Cross-Entropy, vgl. Wikipedia): \[E_{-x} = - \sum_{k}[t_k^x \log(o_k^x) + (1-t_k^x) \log (1- o_k^x)]\] wobei $x$ der Feature-Vektor ist, $k$ ein Neuron des letzen Layers, $t$ der wahre Wert (d.h. der gewünschte Output), $o$ der tatsächliche Output ist.

Stochastic Gradient Descent
Batch Gradient Descent

V07: Feature Learning

Slide name: V07_12-05-2015_Feature_Learning.pdf

Autoencoder (vgl. Wikipedia): Ein Autoencoder ist ein neuronales Netz, welches darauf trainiert wird die Input-Daten am Output wieder zu replizieren.
Bottleneck Features: Unter Bottleneck-Features versteht man eine Schicht in einem neuronalem Netz, welche wesentlich kleiner ist als die vorhergehende und nachfolgende Schicht.
Kullback-Leibler-Divergenz (vgl. Wikipedia): Die Kullback-Leibler-Divergenz ist ein Maß für die Unterschiedlichkeit zweier Wahrscheinlichkeitsverteilungen $P, Q$. Für diskrete Verteilungen ist sie definiert als: \[KL(P||Q) := \sum_{x \in X} P(x) \cdot \log \frac{P(x)}{Q(x)}\]
Denoising Autoencoder: Ein Autoencoder, welcher trainiert wird rauschen zu entfernen.

Fakten:

Eine lineare Aktivierungsfunktion wird in einer Repräsentation im Bottleneck-Feature resultieren, die PCA ähnelt.
Fehlerfunktion:
- CE bei binären Ausgaben (d.h. Input-Features)
- MSE bei reelen Ausgaben (d.h. Input-Features)

Fragen:

Wie muss man die Grafik zu Stacked Denoising Autoencodern verstehen?

V08: Deep Learning

Slide name: V08_2015-05-13_Deep_Learning.pdf

Deep Neural Networks: Neural Networks with at least two hidden layers with nonlinear activation functions.
Hyperparameter: Hyperparameter $\theta$ eines neuronalen Netzes sind Parameter, welche nicht gelernt werden.
Learnin Rate Scheduling: Start with a learning rate $\eta$ and reduce it while training
Exponential Decay Learning Rate: $\eta_t = \eta_{t-1} \cdot \alpha = \eta_0 \cdot \alpha^t$ mit $\alpha \in (0, 1)$
Performance Scheduling: Measure the error on the cross validation set and decrease the learning rate when the algorithm stops improving.
RProp (siehe Wikipedia): RProp is a learning rate scheduling method which is only based on the sign of the gradient. It increases the learning rate when the sign of the gradient doesn't change and increases it when the sign of the gradient changes.
AdaGrad (siehe Wikipedia): \[\eta_{tij} = \frac{\eta_0}{\sqrt{1 + \sum_k {(\frac{\partial E^{t-k}}{\partial w_{ij}})}^2}}\]
Newbob Scheduling: Newbob scheduling is a combination of Exponential decay learning rate scheduling and performance scheduling. It starts with a learning rate $\eta_0$. When the validation error stops decreasing, switch to exponentially decaying learning rate. Terminate when the validation error stops decreasing again.
Cross Entropy Error function (CE): \[E_{CE}(w) = - \sum_{x \in X} \sum_{k} [t_k^x \log(o_k^x + (1-t_k^x) \log(1-o_k^x))]\] where $w$ is the weight vector, $X$ is the set of training examples (feature vectors), $t_k^x = \begin{cases}1 &\text{if } x \text{ is of class }k\\0&\text{otherwise}\end{cases}$ and $o_k^x$ is the output at neuron $k$ of the network for the feature vector $x$.
Mean Squared Error function (MSE): \[E_{MSE}(w) = \frac{1}{2}\sum_{x \in X} \sum_{k} (t_k^x - o_k^x)^2\] where $w$ is the weight vector, $X$ is the set of training examples (feature vectors), $t_k^x = \begin{cases}1 &\text{if } x \text{ is of class }k\\0&\text{otherwise}\end{cases}$ and $o_k^x$ is the output at neuron $k$ of the network for the feature vector $x$.

Pretraining
TDNNs CNNs
Design choices (hyperparameters):
- Topology (Width of layers, number of layers)
- Activation functions
- Error function
- Mini-Batch size
- Training function
- Preprocessing
- Initial Weights
MSE vs CE:
- MSE penetalizes large differences much more than small ones
- MSE works well for function approximation
- CE works well on classification tasks

Fragen:

AdaGrad (Folie 34)
Optimal drain damage (Folie 36)

V09: Reinforcement Learning

Slide name: V09_2015-05-26-Reinforcement-Learning.pdf

Markov Decision Process (MDP, vgl. Wikipedia)

Ein Markovscher Entscheidungsprozess ist ein 5-Tupel $(S, A, T, r, p_0)$, wobei

$S$ eine endliche Zustandsmenge,
$A$ eine endliche Menge von Aktionen,
$T_a(s, s') = T(s_{t+1}=s'|s_t = s, a_t = a)$ die Wahrscheinlichkeit zu einem beliebigen Zeitpunkt von Zustand $s$ mit der Aktion $a$ in den Zustand $a'$ zu kommen (engl. Transition),
$r_a(s, s')$ ist die Belohnung (Reward), die man direkt erhält wenn man erhält wenn man von Zustand $s$ mit Aktion $a$ in Zustand $s'$ kommt,
$p_0$ ist die Startverteilung auf die Zustände $S$

Diskontierungsfaktor

Ein Diskontierungsfaktor $\gamma \in [0, 1]$ encodiert den Bedeutungsverlust zwischen einer direkten Belohnung und einer späteren Belohnung. Es sollte $\gamma < 1$ gelten um unendliche Belohnungen zu vermeiden.

Strategie (engl. Policy)

Eine Strategie $\pi:S \rightarrow A$ sagt einem Agenten welche Aktion er in welchem Zustand ausführen soll.

Q-Funktion (Action-Value function)

Die Q-Funktion $Q^\pi: S \times A \rightarrow \mathbb{R}$ weißt jeder Aktion in jedem Zustand einen Wert zu unter der Annahme, dass die Strategie $\pi$ genutzt wird.

V-Funktion (State-Value function)

Die V-Funktion $V^\pi: S \rightarrow \mathbb{R}$ weißt jeder jedem Zustand die Erwartete Belohnung zu unter der Annahme, dass die Strategie $\pi$ genutzt wird.

$\varepsilon$-Greedy Strategy

Explore $\varepsilon$% of the time. Otherwise, follow what you currently believe is best.

$\varepsilon$-decreasing Strategy

Explore $\varepsilon$% of the time. Otherwise, follow what you currently believe is best. Reduce $\varepsilon$ over time.

$\varepsilon$-first Strategy

Explore for $\varepsilon$ steps and then do what you think is best.

Adaptive $\varepsilon$-greedy Strategy

Explore $\varepsilon$% of the time. Otherwise, follow what you currently believe is best. Reduce $\varepsilon$ based on what you learn.

Episode

A run through an MDP from a start state to an end state.

Monte Carlo Policy Evaluation

Initialize state values $V^\pi$ and iterate:

Generate an episode
foreach state $s$ in episode:
1. Get the reward $\hat{R}$ from that state on
2. $\hat{R} = \sum_{j=0}^\infty \gamma^j r_j$
3. $V_{k+1}^\pi (s) \leftarrow V_k^pi (s) (1-\alpha)+\alpha \hat{R}$

where $\alpha$ is the learning rate.

Temporal Difference Learning (vgl. Wikipedia)

TODO?

Konvention:

Eine optimale Strategie wird mit $\pi^*$ bezeichnet.

Fragen:

Was bedeutet es, wenn in einem MDP der Diskontierungsfaktor (\gamma = 0) ist? → Nur der aktuelle Reward ist wichtig. Effektiv nimmt der Agent immer das nächste Feld, welche den höchsten Reward bietet (bzw. die Aktion, die den größten 1-Aktion Erwartungswert liefert).
Was bedeutet es, wenn in einem MDP der Diskontierungsfaktor (\gamma = 1) ist? → Der Agent versucht die Summe der Belohnungen insgesamt zu maximieren.

TODOs:

What is Policy Iteration?
SARSA - State-Action-Reward-State-Action
Q-Learning

V10: SOM

Slide name: V10_2015-05-26_SOM.pdf

Selbstorganisierende Karten (SOM, Kohonennetze, vgl. Wikipedia): SOMs sind eine Art von Neuronalen Netzen.

Siehe auch:

V11: RBMs

Slide name: V11_2015-05-27_RBMs

Hopfield-Netz (vgl. Wikipedia): Ein Hopfield-Netz besteht nur aus einer Schicht von McCulloch-Pitts Neuronen. Jedes Neuron ist mit jedem anderen Neuron (also nicht sich selbst) und allen Inputs verbunden. Die Schicht funktioniert gleichzeitig als Ein- und Ausgabeschicht.
Boltzmann-Maschine (vgl. Wikipedia): Boltzmann-Maschinen sind stochastische neuronale Netzwerke, welche duch belibige ungerichtete Graphen repräsentiert werden können. Die neuronen sind binär; sie feuern also entweder oder nicht. Es gibt insbesondere keine Unterschiede in der Stärke mit der sie feuern.
Restricted Boltzmann machine (RBM, vgl. Wikipedia): Im Gegensatz zur Boltzmann-Maschine muss die Restricted Boltzmann-Machine (RBM) aus einem bipartitem Graph bestehen. Dies erlaubt ein effizienteres Trainingsverfahren.
Simulated annealing (vgl. Wikipedia): Simulated annealing ist ein heuristisches Optimierungsverfahren. Sei $D$ ein Wertebereich einer Funktion $f: D \rightarrow \mathbb{R}$ und $U: D \rightarrow \mathcal{P}(D)$ eine Funktion, welche die Umgebung eines Punktes angibt. Sei $T: \mathbb{N}_0 \rightarrow \mathbb{R}_{> 0}$ die Temperatur zum Zeitpunkt $t \in \mathbb{N}_0$. Gesucht ist $\text{arg min}_{x \in D} f(x)$. Wähle zum Zeitpunkt $t=0$ einen zufälligen Startwert $x \in D$. Gehe nun iterativ vor und jeweils einen Zeitschritt weiter: Nehme einen Punkt aus der Umgebung $y \in U(x)$. Wenn $f(y) \leq f(x)$, dann überschreibe $x \leftarrow y$. Falls nicht, dann überschreibe es mit der Wahrscheinlichkeit $\exp \left (-\frac{f(y)-f(x)}{T(t)} \right )$. Speichere in jedem Schritt den bisher besten Wert.

Anwendungen:

RBMs:
- Collaborative Filtering: User-rating prediction for movie database. The problem is that not every user has rated all movies.

Fragen:

Hopfield-Netze:
- Wie trainiert man sie?
- Wo werden / wurden sie benutzt?

Siehe auch:

Deeplearning.net: Restricted Boltzmann Machines (RBM)
Wikipedia: Restricted Boltzmann machine

V12: RNNs

Slide name: V12_2015-06-02_RNNs.pdf

Elman-Netz (vgl. Wikipedia): Ein rekurrentes neuronales Netzwerk, bei dem die Ausgabe eines hidden layers im nächsten Zeitschritt als Eingabe verwendet wird.
Jordan-Netz (vgl. Wikipedia): Ein rekurrentes neuronales Netzwerk, bei dem die Ausgabe der Ausgabeschicht im nächsten Zeitschritt als Eingabe verwendet wird.
Backpropagation through Time (BPTT): Ein Trainingsalgorithmus für rekurrente neuronale Netze, bei dem das Netz "ausgerollt" wird. Das rekurrente Netz wird also als unendlich großes nicht-rekurrentes Netz behandelt.
Vanishing gradient problem (vgl. Wikipedia): Das Problem des verschwindenden Gradienten ist eine Herausforderung im Kontext neuronaler Netze, welche mit Backpropagation trainiert werden. Insbesondere bei sehr tiefen oder rekurrenten Netzen kann es passieren, dass der Gradient bei den ersten Schichten sehr niedrig ist, sodass das Netz sehr langsam lernt. Aufgrund numerischer Ungenauigkeit kann dies sogar dazu führen, dass das Netz in den ersten Schichten nicht lernen kann.
Long short-term memory (vgl. LSTM): Ein LSTM ist ein Typ eines neuronalen Netzwerks. Das besondere an LSTM Netzen sind "intelligente" Neuronen, welche über Gates bestimmen ob ein Wert gespeichert wird und wie lange.

Siehe auch:

V13: NNlearning-tricks

Slide name: V13_2015-06-09_NNlearning-tricks.pdf

Momentum: In der Update-Regel $\Delta w_{ij}^* (t+1) = \Delta w_{ij} (t+1) + \alpha \Delta w_{ij}(t)$ wird der Term $\Delta w_{ij}(t)$ als Momentum bezeichnet. Der Skalar $\alpha \in [0, 1]$ gewichtet diesen und ist ein Hyperparameter.
Quickprop (Wikipedia): Quickprop ist ein Trainingsverfahren für neuronale Netze. TODO: Wie funktioniert es?
Weight Decay: Passe die Fehlerfunktion an: $E = MSE + \lambda \sum_{i,j} w_{ij}^2$
Weight Elimination: Passe die Fehlerfunktion an: $E = MSE + \lambda \sum_{i,j} \frac{w_{ij}^2}{1+w_{ij}^2}$
Optimal Brain Damage: Optimal Brain Damage entfernt nach dem Training Verbindungen die sehr kleine $|w_{ij}|$ haben. Besser: Entferne Verbindungen, die geringen Einfluss auf die Fehlerfunktion haben.
Cascade Correlation Architecture (siehe Fahlman und Lebiere: The Cascade-Correlation Learning Architecture): Die Cascade Correlation Architecture wird Schritt für Schritt aufgebaut. Sie ist keine typische Multilayer-Architektur.

Speed-ups sind möglich durch:

Momentum
Überspringen von bereits gut gelernten Beispielen
Dynamische Anpassung der Lernrate $\eta$
Quickprop
Gute Initialisierung

Lernen kann getweakt werden:

Fehlerfunktion anpassen
- MSE
- Cross-Entropy
- CFM
Overfitting verhindern
- Weight decay
- Weight elimination
- Optimal Brain Damage
- Optimal Brain Surgeon
Schrittweise Netzkonstruktion
- Cascade Correlation
- Meiosis Netzwerke (siehe Stephen Jose Hanson: Meiosis Networks )
- ASO: TODO - wie funktioniert das?

Fragen:

Folie 19: Was passiert hier? (TODO)

V14: DNN CV

Slide name: V14_2015-06-10_DNN_CV .pdf

SIFT (Scale-invariant feature transform, siehe Wikipedia): Unter SIFT versteht man bestimmte Features in der Bildverarbeitung, welche invariant unter skalierung sind.
Texton (siehe Wikipedia und UCLA): Unter einem Texton versteht man grundlegende, kleine Features eines Bildes. Diese Bilden die kleinsten als unterschiedlich wahrnehmbaren Einheiten.
Convolutional Neural Network (CNN, siehe Wikipedia): Ein CNN ist ein Neuronales Netzwerk, welches mindestens eine Schicht hat, welche die Parameter eines Kernels für eine Faltung lernt.
Feature Map: Im Kontext von CNNs versteht man unter einer Feature-Map die Ausgabe eines Kernels in einem Convolutional Layer.

Facts:

Pooling: Max, Mean, Probabilistic

V15: Speech-Independence

Slide name: V15_2015-06-17_Speech-Independence.pdf

Visualisierung von Netzen

Häufig wird die Architektur neuronaler Netze grafisch dargestellt. Dabei ist mir folgendes aufgefallen:

Im Innenren von Neuronen wird die Aktivierungsfunktion “geplottet”. Das heißt bei der Sigmoidfunktion wird etwas S-Förmiges dargestellt, bei der sign-Funktion etwas eckiges, bei ReLU ein horizontaler Strich gefolgt von einem Strich im 45-Grad Winkel … (TODO: Beispiele aufzeichnen)
Typischerweise ist der Input links (oder alternativ unten) und der Output rechts (oder alternativ oben)

Material und Links

Vorlesungswebsite
NNPraktikum: Toolkit für die Übungsblätter
StackExchange
Visualizing Optimization Algos
Neural Network demo
Skript von Marvin Ritter
Machine Learning 1 und Machine Learning 2 am KIT
Coursera: Neural Networks for Machine Learning by Geoffrey Hinton

Übungsbetrieb

Übungsblätter sind freiwillig.

Termine und Klausurablauf

Datum: nach Terminvereinbarung
Ort: Gebäude 50.20
Übungsschein: gibt es nicht
Bonuspunkte: gibt es nicht
Erlaubte Hilfsmittel: keine

Mustererkennung - Klausur

info@martin-thoma.de (Martin Thoma) — Mon, 27 Apr 2015 21:15:00 +0200

Dieser Artikel beschäftigt sich mit der Vorlesung „Mustererkennung“ am KIT. Er dient als Prüfungsvorbereitung. Ich habe die Vorlesungen bei Herrn Prof. Dr.-Ing. Jürgen Beyerer im Sommersemester 2015 gehört und einige Abschnitte direkt aus den Folien übernommen.

Behandelter Stoff

Vorlesung

Datum	Kapitel	Inhalt
15.04.2015	Einleitung	`$\hat{w}$` - das `^` bedeutet, dass die Klasse geschätzt ist.
22.04.2015	Kapitel 2 - Merkmale: 1-31?	Welt; Domäne; Objekte; Klassen; Merkmalsraum; Merkmalsvektor; Klassifikation; Skalen (nominal, ordinal, intervall-, verhältnis- und absolutskaliert); Projektionen; Norm (Minkowski, Euklidisch, Chebychev, Mahalanobis); Metrik (Tanimoto)
14.07.2015	Kapitel 8 - Klassifikatoren (1-28), Kapitel 9: 1-?	Entscheidungsbäume, Grammatiken; Lernen nach Vapnik, VC-dimension, Kreuzvalidierung und Leave-One-Out, Boosting

Folien

ME-Kap1_V31.pdf

Einleitendes Kapitel welches erklärt, was Klassifikation ist.

Beispiele für Klassifikation: Blumen/Schmetterlinge in Arten; Schrauben in Schraubentypen; Schüttgut in Mineralien, Pflanzen, Glasscheiben, Diamante, …
Formalismen
- Domäne $\Omega \subseteq $ Welt, Elemente der Domäne heißen Objekte, Objekte werden in paarweise disjunkte Äquivalenzklassen $\omega_i$ gruppiert, sodass jedes Objekt genau eine Äquivalenzklasse hat.
- Man beobachtet / misst Eigenschaften realer Objekte. Dies kann als Funktion m aufgefasst werden, die von der Domäne in den Merkmalsraum abbildet. Optimalerweise ist diese Abbildung injektiv, bei ungünstig gewählten Merkmalen jedoch nicht. Klassifikatoren arbeiten auf dem Merkmalsraum und finden eine Partition des Merkmalsraumes in Klassen
Muster: Gesamtheit der beobachteten / gemessenen Werte einer einzelnen Stichprobe (eines einzelnen Objekts).
Erkennung: (Wieder)erkennung von etwas, was bereits bekannt ist.
Merkmale: eruirbare, charakteristische Eigenschaften, die als Basis für die Untersuchung von Mustern dienen soll.
Mustererkennungsschritte: Sensierung ergibt Muster; Vorverarbeitung; Segmentierung; Merkmalsextraktion ergibt Merkmale; Klassifikation ergibt Äquivalenzklassen
Überwachtes lernen: Vorklassifizierte Beispiele sowie die Klassenstruktur sind gegeben; eventuell auch Auftrittswahrscheinlichkeiten $P(\omega_i)$ der Klassen
Gesamtstichprobe wird in die disjunkten Mengen Lernstrichprobe, Validierungsstichprobe und Teststichprobe zerlegt.

ME-Kap2_V84.pdf

In diesem Foliensatz geht es um Merkmale und ihre Eigenschaften.

	Skala
	qualitativ		quantitativ (metrisch)
	Nominal-	Ordinal-	Intervall-	Verhältnis-	Absolut
Empirische Relation	~ Äquivalenz	~ Äquivalenz Ordnung	~ Äquivalenz Ordnung Emp. Addition	~ Äquivalenz Ordnung Emp. Addition Emp. Multipliation	~ Äquivalenz Ordnung Emp. Addition Emp. Multipliation
Zulässige Transformationen	m' = f(m) f bijektiv	m' = f(m) f streng monoton	m' = am+b mit a>0	m' = am mit a>0	m' = m
Beispiele zugehörige Merkmale	Telefonnummern, Kfz-Kennz., Typen, PLZ, Geschlecht	Güteklassen, Härtegrad, Windstärke	Temp. in °C, °F, Kalenderzeit, geographische Höhe	Masse, Länge, el. Strom	Quantenzahlen, Teilchenanzahl, Fehlerzahl
Werte von m	Zahlen, Namen, Symbole	in der Regel natürliche Zahlen	in der Regel reele Zahlen	in der Regel reele Zahlen > 0	in der Regel natürliche Zahlen

Der Merkmalsraum ist häufig ein $\mathbb{R}^n$ mit $n>3$ . Er kann auf vorhandene Strukturen analysiert werden, indem er auf einen 2- oder 3-dimensionalen unterraum projeziert wird. Dies kann bei einfachen Projektionen jedoch nicht erfolgreich sein, wenn beispielsweise zwei Klassen Schalenförmig um den Urspruch angeordnet sind.

Um Stichproben im Merkmalsraum zu vergleichen können Metriken benutzt werden. Eine Metrik ist eine Abbildung $d(m_1, m_2)$ , für die gilt:

Positive Definitheit: $d(m_1, m_2) \geq 0$ und $d(m_1, m_2) = 0 \Leftrightarrow m_1 = m_2$ ,
Symmetrie: $d(m_1, m_2) = d(m_2, m_1)$ ,
Dreiecksungleichung: $d(m_1, m_2) \leq d(m_1, m_3) + d(m_3, m_2)$

Metriken können durch Normen erzeugt werden, indem $d(m_1, m_2) := \|m_1 - m_2\|$ definiert wird. Eine Norm ist eine Abbildung $\| \cdot \|: V \rightarrow \mathbb{R}_0^+, x \mapsto \|x\|$ für die gilt:

Definitheit: $\|x\| = 0 \Rightarrow x = 0$
Absolute Homogenität: $\|\alpha \cdot x \| = \alpha \cdot \| x \|$
Dreiecksungleichung: $\|x+y\| \leq \|x\| + \|y\|$

Typische Normen sind die euklidische Norm und die Mahalanobis Norm $\|m\| := \sqrt{m^T A m}$ mit $A$ positiv definit.

Hauptkomponentenanalyse (HKA, engl. PCA)

Finde $m_0$ , sodass $J_0(m) := \sum_{k=1}^N \|m - m_k\|^2$ minimal ist, also $m_0 = \frac{1}{N} \sum_{k=1}^N m_k$
Finde Gerade $h: m = \bar{m} + ae$ , welche die Punkte optimal repräsentiert.
1. Finden der $a_k$ (TODO: Was ist das?) Fehlermaß $J_1(a_1, \dots, a_N, e) = \sum_{k=1}^N \|\bar{m} + a_k e - m_k \|^2$ . Ergibt: $a_k = e^T (m_k - \bar{m})$
2. Berechnung des optimalen Richtungsvektors Streumatrix $S := \sum_{k=1}^N (m_k - \bar{m}) (m_k - \bar{m})^T$
Finden eines affinen $d'$ -dimensionalen Unterraumes des Merkmalsraumes, welcher die Daten $D$ mit minimalen quadratischem Fehler repräsentiert.

Siehe gist für eine kurze Python-Implementierung. Keine Garantie für die Korrektheit!

Kernelized PCA
Independent Component Analysis (ICA)
Multiple Discriminant Analysis (MDA)

ME-Kap3_V52.pdf

Bayessche Klassifikatoren wählen die Klasse aus, die die größte Wahrscheinlichkeit besitzt. Dazu verfolgt man den Ansatz

$$P(\omega|m) = \frac{p(m|\omega) \cdot P(\omega)}{p(m)}$$

Dabei wird $P(\omega|m)$ die A Posteriori Wahrscheinlichkeitsverteilung und $P(\omega)$ die A Priori Wahrscheinlichkeitsverteilung genannt.

ME-Kap4_V33.pdf

Parameterschätzung kann entweder mit der Likelihood-Methodik oder mit der Bayesschen Methodik durchgeführt werden. Die Idee der Likelihood-Methodik ist es, den Parameter $\theta$ als unbekannte konstante (d.h. nicht-stochastische) Größe anzusehen. Man wählt $\theta$ also so, dass die Wahrscheinlichkeit der Beobachtungen gegeben $\theta$ maximiert wird.

Die Bayessche Methodik geht dagegen davon aus, dass $\theta$ auch eine Zufallsvariable ist und über eine Wahrscheinlichkeitsverteilung beschrieben werden kann.

Schätzer können verschiedene Qualitätskriterien erfüllen, z.B. Erwartungstreue oder Konsistenz.

Bei der Parameterschätzung können folgende Fehler passieren:

Bayesscher Fehler: (TODO: Was ist das?)
Modellfehler: Unpassendes Modell gewählt (Falsche Verteilungsannahme?)
Schätzfehler: Zu wenige Daten um Parameter korrekt zu bestimmen

ME-Kap5_V31.pdf

Parameterfreie Methoden heißen “parameterfrei”, weil sie keine konkrete Wahrscheinlichkeitsverteilung parametrisieren und den Parameter schätzen. Die Parameterfreien Methoden können sehr wohl Parameter benutzen. Beispiele sind:

ME-Kap6_V18.pdf

Allgemeine Problemstellungen:

Dimension des Merkmalsraumes
Overfitting

ME-Kap7_V54.pdf

Spezielle Klassifikatoren:

Lineare Diskriminanzfunktionen: Linear bezieht sich hier auf die Kombination der Merkmale. Man kann allerdings Merkmale wählen, die z.B. das quadrat eines gemessenen wertes sind.
Perzeptron
Lineare Regression
Künstliche Neuronale Netze
Support Vector Machines (SVMs)
Matched Filter
HMMs (Sequenzen)
Klassifikation mit Rückweisung (Maximum / Minimum / Differenz / Abstand)

ME-Kap8_V21.pdf

Klassifikation bei nominalen Merkmalen:

Entscheidungsbäume
String-Verfahren
Grammatiken

ME-Kap9_V27.pdf

Klassifikatorunabhängige Prinzipien:

Generalisierung / Generalisierungsfähigkeit
VC-Konfidenz / VC-Dimension
Structural Risc Minimization
Kreuzvalidierungsverfahren / Leave-one-out
Boosting

Prüfungsfragen

Warum ist ein hochdimensionaler Merkmalsraum schlecht (curse of dimensionality)?
- Je nach Klassifikator, viele zu lernende Parameter
- Daten haben einen sehr hohen Abstand zueinander → Gefahr des Overfittings
Wie kann man die Dimension des Merkmalsraumes reduzieren? → Merkmalsauswahl, suboptimales iteratives Verfahren, HKA (Varianzen maximieren), MDA (Klassentrennbarkeit maximieren), ICA
Wie viele Möglichkeiten gibt es 5 Merkmale aus 10 auszuwählen? → Binomialkoeffizient
Was ist Overfitting? (Zu starke Anpassung des Klassifizierers an die Lerndaten; geringe Generalisierungsfähgikeit)
Welche Probleme gibt es, wenn man Länge, Masse und Temperatur als Merkmale hat?
- Unterschiedliche Einheiten (→ Entdimensionalisieren)
- Unterschiedliche Skalen (→ Teilen durch Varianz oder durch Wertebereich)
- Unterschiedliche Wertebereiche (→ Durchschnitt abziehen)
Wie funktioniert MDA? → Sie maximiert $J(w) = \frac{|m'_1 - m'_2|^2}{s'_1^2 - s'_2^2}$ (im 2-Klassen Fall, wobei $w$ die Ebene ist, auf die projeziert wird)
Wie unterscheidet sich PCA/MDA von dem suboptimalen Algorithmus zur Merkmalsauswahl? → PCA/MDA sind Klassifikatorunabhängig, aber der suboptimale Algorithmus benötigt bereits einen Klassifikator.
Wie lautet die Fundamentalformel der Bayesschen Klassifikation? → $P(A|B) = \frac{P(A)\, P(B | A)}{P(B)}$ (wobei üblicherweise B das Merkmal ist und A die Klasse)
Wie lautet die Hauptformel der PCA? $m' = A^T \cdot (m - \bar{m})$ , wobei $A$ die Basiswechselmatrix ist.
Wie kann man invariante Merkmale erzeugen? → Integration über eine Transformationsgruppe, Differentielle Methode, Normalisierung
Wie kann man normalisieren? → Fourierdeskriptoren kann man invariant bzgl. Translation und Rotation und radialer Streckung (Skalierung) machen
Wie lauten die Prinzipien (A) - (E) der SVMs?
- (A) Lineare Trennung mit maximalen Abstand der Trennebenen zu den nächstgelegenen Stichproben (Support Vektoren)
- (B) Duale Formulierung des linearen Klassifikators. (vgl. Wiki, $k(m) = w^T m + b = \langle w, m \rangle + b = \sum_{j=1}^N \alpha_j z_j \langle m_j, m \rangle + b$ )
- (C) Nichtlineare Abbildung der primären Merkmale in einen hochdimensionalen Merkmalsraum $\Phi$
- (D) Implizite Nutzung des unter Umständen $\infty$ -dimensionalen Eigenfunktionsraumes einer sog. Kernfunktion $K$ als transformierten Merkmalsraum $\Phi$ . Dabei müssen die transformierten Merkmale nicht explizit berechnet werden und der Klassifikator hat trotz der hohen Dimension von $\Phi$ nur eine niedrige Zahl von freien Parametern (Kernel-Trick).
- (E) Relaxation der Forderung nach linearer Trennbarkeit durch Einführung von Schlupfvariablen (slack variables).
Wie lautet die Dichtefunktion der $d$ -dimensionale Gaußverteilung? $f_X(x) = \frac{1}{\sqrt{(2\pi \det{\Sigma})}} \exp(-\frac{1}{2}(x-\mu)^T \Sigma^{-1} (x-\mu))$
Wie lautet Mercers Theorem? → wiki
Wie ist die Kullback-Leibler-Divergenz defininiert?

Material und Links

Vorlesungswebsite: Ist passwortgeschützt. Das Passwort (das ausnahmsweise mal nicht zu erraten ist) kann ich hier natürlich nicht schreiben. Aber der Benutzername ist asbstudent.
SVMs
- Why bother with the dual problem when fitting SVM?
- A Tutorial on Support Vector Machines for Pattern Recognition

Übungsbetrieb

Es gibt keine Übungsblätter, keine Übungen, keine Tutorien und keine Bonuspunkte.

Termine und Klausurablauf

Datum: Donnerstag, der 10.09.2015 von 11:00-13:00 Uhr (Quelle: Wurde in der Vorlesung vom 22.04.2015 gesagt)
Ort: Gerthsen-Hörsal
Punkte: 90
Zeit: 90 min
Punkteverteilung: ?

ab 60.5: 1.7

Bestehensgrenze: ?
Übungsschein: gibt es nicht
Bonuspunkte: gibt es nicht
Ergebnisse: Am 30.09.2015 war die (vorläufige) Note im Notenauszug
Einsicht: Montag 12.10.2015, 9:00-15:00 Uhr im Geb. 50.21, Raum 015.1
Erlaubte Hilfsmittel: keine

Notenverteilung

Wenn ihr mir schreibt was ihr habt, kann ich das updaten:

1,3: min 1
2,0: min 1

Lasagne for Python Newbies

info@martin-thoma.de (Martin Thoma) — Fri, 17 Apr 2015 19:26:00 +0200

Lasagne is a Python package for training neural networks. The nice thing about Lasagne is that it is possible to write Python code and execute the training on nVidea GPUs with automatically generated CUDA code.

However, installing Lasagne is not that easy. Especially if you are not familiar with Python. This article aims to guide you through the installation process.

Python

Ubuntu-based systems will have Python installed, but I’m not too sure about pip. You can get it with

$ sudo apt-get install python-pip

Make sure you have Python and pip, the standard Python package installer. Type the following commands to check if you have both:

$ python --version
Python 2.7.8

$ pip --version
pip 6.1.1 from /usr/local/lib/python2.7/dist-packages (python 2.7)

sklearn

sklearn is a nice package for machine learning. You can install it with

$ pip install scikit-learn

(When I write commands like this you either have to execute sudo pip install scikit-learn or pip install scikit-learn --user).

This should work without problems.

Each classifier has a fit method and a predict method. See iris example to get a feeling how to use it. It provides a lot of useful functions like train_test_split and has an awesome documentation.

You don’t need this for Lasagne, but it might be good to use sklearn and Lasagne in combination.

Graphics drivers and CUDA

Make sure CUDA runs on your system by the following commands. If it doesn’t run, you could try the following guides:

Run

$ nvidia-smi -L
GPU 0: GeForce GTX TITAN Black (UUID: GPU-abcdef12-abcd-1234-1234-01234567890a)

$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2013 NVIDIA Corporation
Built on Thu_Mar_13_11:58:58_PDT_2014
Cuda compilation tools, release 6.0, V6.0.1

$ nvidia-smi -a

==============NVSMI LOG==============

Timestamp                           : Fri Apr 17 18:44:41 2015
Driver Version                      : 331.79

Attached GPUs                       : 1
GPU 0000:01:00.0
    Product Name                    : GeForce GTX TITAN Black
    Display Mode                    : N/A
    Display Active                  : N/A
    Persistence Mode                : Disabled
    Accounting Mode                 : N/A
    Accounting Mode Buffer Size     : N/A
    Driver Model
        Current                     : N/A
        Pending                     : N/A
    Serial Number                   : N/A
    GPU UUID                        : GPU-fcff168f-a045-2f95-7a4f-8e1cf26a24eb
    Minor Number                    : 0
    VBIOS Version                   : 80.80.4E.00.01
    Inforom Version
        Image Version               : N/A
        OEM Object                  : N/A
        ECC Object                  : N/A
        Power Management Object     : N/A
    GPU Operation Mode
        Current                     : N/A
        Pending                     : N/A
    PCI
        Bus                         : 0x01
        Device                      : 0x00
        Domain                      : 0x0000
        Device Id                   : 0x100C10DE
        Bus Id                      : 0000:01:00.0
        Sub System Id               : 0x106610DE
        GPU Link Info
            PCIe Generation
                Max                 : N/A
                Current             : N/A
            Link Width
                Max                 : N/A
                Current             : N/A
        Bridge Chip
            Type                    : N/A
            Firmware                : N/A
    Fan Speed                       : 26 %
    Performance State               : N/A
    Clocks Throttle Reasons         : N/A
    FB Memory Usage
        Total                       : 6143 MiB
        Used                        : 39 MiB
        Free                        : 6104 MiB
    BAR1 Memory Usage
        Total                       : N/A
        Used                        : N/A
        Free                        : N/A
    Compute Mode                    : Default
    Utilization
        Gpu                         : N/A
        Memory                      : N/A
    Ecc Mode
        Current                     : N/A
        Pending                     : N/A
    ECC Errors
        Volatile
            Single Bit
                Device Memory       : N/A
                Register File       : N/A
                L1 Cache            : N/A
                L2 Cache            : N/A
                Texture Memory      : N/A
                Total               : N/A
            Double Bit
                Device Memory       : N/A
                Register File       : N/A
                L1 Cache            : N/A
                L2 Cache            : N/A
                Texture Memory      : N/A
                Total               : N/A
        Aggregate
            Single Bit
                Device Memory       : N/A
                Register File       : N/A
                L1 Cache            : N/A
                L2 Cache            : N/A
                Texture Memory      : N/A
                Total               : N/A
            Double Bit
                Device Memory       : N/A
                Register File       : N/A
                L1 Cache            : N/A
                L2 Cache            : N/A
                Texture Memory      : N/A
                Total               : N/A
    Retired Pages
        Single Bit ECC              : N/A
        Double Bit ECC              : N/A
        Pending                     : N/A
    Temperature
        Gpu                         : 29 C
    Power Readings
        Power Management            : N/A
        Power Draw                  : N/A
        Power Limit                 : N/A
        Default Power Limit         : N/A
        Enforced Power Limit        : N/A
        Min Power Limit             : N/A
        Max Power Limit             : N/A
    Clocks
        Graphics                    : N/A
        SM                          : N/A
        Memory                      : N/A
    Applications Clocks
        Graphics                    : N/A
        Memory                      : N/A
    Default Applications Clocks
        Graphics                    : N/A
        Memory                      : N/A
    Max Clocks
        Graphics                    : N/A
        SM                          : N/A
        Memory                      : N/A
    Compute Processes               : N/A

to see if CUDA was installed correctly.

Theano

The installation of Theano is a bit tricky (see official page). I don’t remember if I installed additional packages, but try

$ sudo -H pip install theano

Make sure that your ~/.theanorc exists and looks like this:

[global]
device=gpu
floatX=float32

Note that float32 is required, even if you have a 64bit system.

To test your installation, save the following as theanotest.py and execute it with python theanotest.py:

#!/usr/bin/env python
from theano import function, config, shared, sandbox
import theano.tensor as T
import numpy
import time

vlen = 10 * 30 * 768  # 10 x #cores x # threads per core
iters = 1000

rng = numpy.random.RandomState(22)
x = shared(numpy.asarray(rng.rand(vlen), config.floatX))
f = function([], T.exp(x))
print f.maker.fgraph.toposort()
t0 = time.time()
for i in xrange(iters):
    r = f()
t1 = time.time()
print 'Looping %d times took' % iters, t1 - t0, 'seconds'
print 'Result is', r
if numpy.any([isinstance(x.op, T.Elemwise) for x in f.maker.fgraph.toposort()]):
    print('Used the cpu')
else:
    print('Used the gpu')

It should print the following (well, something similar):

Using gpu device 0: GeForce GTX TITAN Black
[GpuElemwise{exp,no_inplace}(<CudaNdarrayType(float32, vector)>), HostFromGpu(GpuElemwise{exp,no_inplace}.0)]
Looping 1000 times took 0.38205909729 seconds
Result is [ 1.23178029  1.61879349  1.52278066 ...,  2.20771813  2.29967761
  1.62323296]
Used the gpu

Especially “used the gpu” is important. Theano code work on both, CPU and GPU. If you have a GPU and it does not currently work on a task and it is configured correctly, then Theano should automatically use the GPU.

(Don’t try to run two Theano scripts at a time … weird things could happen.)

Lasagne

Lasagne is hosted at Github: https://github.com/Lasagne/Lasagne

Currently, it is not on pip as Sander wants to wait until we get to version 1.0. So you have to install it manually:

$ git clone https://github.com/Lasagne/Lasagne.git
$ cd Lasagne
Lasagne$ sudo -H python setup.py install

Now you can test if it worked by executing the MNIST example in Lasagne (MNIST is a huge digit dataset). This might first take some time to download, but should then run quite fast. If your machine does not use the GPU it will take ages (e.g. on my laptop it takes about a minute for one epoch)

Lasagne/examples$ python mnist.py
Loading data...
Downloading MNIST dataset
Building model and compiling functions...
/usr/local/lib/python2.7/dist-packages/Lasagne-0.1dev-py2.7.egg/lasagne/init.py:30: UserWarning: The uniform initializer no longer uses Glorot et al.'s approach to determine the bounds, but defaults to the range (-0.01, 0.01) instead. Please use the new GlorotUniform initializer to get the old behavior. GlorotUniform is now the default for all layers.
  warnings.warn("The uniform initializer no longer uses Glorot et al.'s "
/usr/local/lib/python2.7/dist-packages/Lasagne-0.1dev-py2.7.egg/lasagne/layers/helper.py:55: UserWarning: get_all_layers() has been changed to return layers in topological order. The former implementation is still available as get_all_layers_old(), but will be removed before the first release of Lasagne. To ignore this warning, use `warnings.filterwarnings('ignore', '.*topo.*')`.
  warnings.warn("get_all_layers() has been changed to return layers in "
Starting training...
Epoch 1 of 500 took 72.593s
  training loss:        1.330231
  validation loss:        0.470251
  validation accuracy:        87.54 %%

nolearn

nolearn is another Python package. It was created to make using Lasagne even simpler.

You can install it via

$ git clone https://github.com/dnouri/nolearn.git
$ cd nolearn
$ python setup.py install --user

For example, the following code downloads the MNIST dataset, trains a model on it and evaluates the result for a single image:

#!/usr/bin/env python

import lasagne
from lasagne import layers
from lasagne.updates import nesterov_momentum
from nolearn.lasagne import NeuralNet

import sys
import os
import gzip
import pickle
import numpy


PY2 = sys.version_info[0] == 2

if PY2:
    from urllib import urlretrieve

    def pickle_load(f, encoding):
        return pickle.load(f)
else:
    from urllib.request import urlretrieve

    def pickle_load(f, encoding):
        return pickle.load(f, encoding=encoding)

DATA_URL = 'http://deeplearning.net/data/mnist/mnist.pkl.gz'
DATA_FILENAME = 'mnist.pkl.gz'


def _load_data(url=DATA_URL, filename=DATA_FILENAME):
    """Load data from `url` and store the result in `filename`."""
    if not os.path.exists(filename):
        print("Downloading MNIST dataset")
        urlretrieve(url, filename)

    with gzip.open(filename, 'rb') as f:
        return pickle_load(f, encoding='latin-1')


def load_data():
    """Get data with labels, split into training, validation and test set."""
    data = _load_data()
    X_train, y_train = data[0]
    X_valid, y_valid = data[1]
    X_test, y_test = data[2]
    y_train = numpy.asarray(y_train, dtype=numpy.int32)
    y_valid = numpy.asarray(y_valid, dtype=numpy.int32)
    y_test = numpy.asarray(y_test, dtype=numpy.int32)

    return dict(
        X_train=X_train,
        y_train=y_train,
        X_valid=X_valid,
        y_valid=y_valid,
        X_test=X_test,
        y_test=y_test,
        num_examples_train=X_train.shape[0],
        num_examples_valid=X_valid.shape[0],
        num_examples_test=X_test.shape[0],
        input_dim=X_train.shape[1],
        output_dim=10,
    )


def nn_example(data):
    net1 = NeuralNet(
        layers=[('input', layers.InputLayer),
                ('hidden', layers.DenseLayer),
                ('output', layers.DenseLayer),
                ],
        # layer parameters:
        input_shape=(None, 28*28),
        hidden_num_units=100,  # number of units in 'hidden' layer
        output_nonlinearity=lasagne.nonlinearities.softmax,
        output_num_units=10,  # 10 target values for the digits 0, 1, 2, ..., 9

        # optimization method:
        update=nesterov_momentum,
        update_learning_rate=0.01,
        update_momentum=0.9,

        max_epochs=10,
        verbose=1,
        )

    # Train the network
    net1.fit(data['X_train'], data['y_train'])

    # Try the network on new data
    print("Feature vector (100-110): %s" % data['X_test'][0][100:110])
    print("Label: %s" % str(data['y_test'][0]))
    print("Predicted: %s" % str(net1.predict([data['X_test'][0]])))


def main():
    data = load_data()
    print("Got %i testing datasets." % len(data['X_train']))
    nn_example(data)

if __name__ == '__main__':
    main()

The output is

# Neural Network with 79510 learnable parameters

## Layer information

  #  name      size
---  ------  ------
  0  input      784
  1  hidden     100
  2  output      10

  epoch    train loss    valid loss    train/val    valid acc  dur
-------  ------------  ------------  -----------  -----------  -----
      1       0.59132       0.32314      1.82993      0.90988  1.70s
      2       0.30733       0.26644      1.15348      0.92623  1.96s
      3       0.25879       0.23606      1.09629      0.93363  2.09s
      4       0.22680       0.21424      1.05865      0.93897  2.13s
      5       0.20187       0.19633      1.02827      0.94313  2.21s
      6       0.18129       0.18187      0.99685      0.94758  1.81s
      7       0.16398       0.16992      0.96506      0.95074  2.14s
      8       0.14941       0.16020      0.93265      0.95262  1.88s
      9       0.13704       0.15189      0.90222      0.95460  2.15s
     10       0.12633       0.14464      0.87342      0.95707  2.21s
Feature vector (100-110): [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]
Label: 7
Predicted: [7]

Echtzeitsysteme - Klausur

info@martin-thoma.de (Martin Thoma) — Tue, 14 Apr 2015 20:43:00 +0200

Dieser Artikel beschäftigt sich mit der Vorlesung „Echtzeitsysteme“ am KIT. Er dient als Prüfungsvorbereitung. Ich habe die Vorlesungen bei Herrn Prof. Dr. Wörn im Sommersemester 2015 gehört.

Behandelter Stoff

Vorlesung

Der Dozent verwendete einige Abkürzungen, die mir nicht geläufig waren. Diese habe ich unter anderem in der folgenden Tabelle aufgeführt.

Datum	Kapitel	Inhalt
14.04.2015	Kapitel 1 (ES1-1 - ES1-24)	SPS; NC; RC; Daisy chain; Mikroprozessor vs. Mikrorechner vs. Mikrorechnersystem; Aufbau eines Mikroprozessors; Speicherhierachie (L1-, L2- und L3-Cache, Hauptspeicher, Festplatte); superskalare Pipeline; Fixed- priority-preemptive Unterbrechungsbehandlung
15.04.2015	Kapitel 1 (ES1-25 - ES1-60)	Watchdogs, Jitter, DMA, Signalprozessor, VLIW; verschiedene Bus-Arten (z.B. PCI-Bus), Bus-Arbitation kümmert sich bei mehreren Bus-Mastern um die Zugriffskontrolle. Das kann zentral oder dezentral (z.B. über eine Daisy-Chain oder einen Identifikationsbus) geschehen;
21.04.2015	Kapitel 1 (ES1-62 - ES1-83)	Arbitrierung; Transaktionen
22.04.2015	Kapitel 2 (ES2-1 - ES2-?)	Ohmsches Gesetz; Parallel- und Reihenschaltung von Widerständen; Spannungsteiler; Knotenregel; Maschenregel; Transistoren; Addierer; Differenzierer; Invertierter Addierer; Integrierer; AD/DA-Wandler
28.04.2015	Kapitel 2 (?)	?
12.05.2015	?	Hurwitz-Kriterium; Ortskurve; P-, Pi- und PID-Regler (I: Rück-Knala, D: Änderung); Vorsteuerung, Car Cruise Control
19.05.2015	2. Übung	Operationsverstärker, AD-Wandler
20.05.2015	?	?
26.05.2015	3. Übung	Mit Jessica Hutzl
27.05.2015	Kapitel 4 (- ES 4-43)	ISO/OSI-Modell, Profibus, CAN-Bus, INTERBUS
02.06.2015	Kapitel 4, 5 (ES 4-44 - 5-16)	Rechtzeitigkeit
03.06.2015	?	?
09.06.2015	Übung	Regelung
10.06.2015	Übung	-

Abkürzungen

LLF: Least-Laxity-First-Scheduling
EDF: Earliest-Deadline-First-Scheduling
GPS: Guaranteed Percentage Scheduling

Wichtiges

Ist eine Übertragungsfunktion in Pol-und-Nullstellenform $G(s)$ stabil? → Ja, falls der Realanteil aller Pole negativ ist.
Ist eine Funktion $G(s)$ stabile? → Ja, falls die Nullstellen der Gleichung $G(s)+1=0$ links der $i$ -Achse liegen.
Ist $G(i \omega)$ stabil? → Ja, falls die Kurve (-1, 0i) NICHT umfährt (siehe Nyquistkriterium)

Hurwitz-Kriterium

Ein System $a_n x^{(n)} + a_{n-1} x^{(n-1)} + \dots + a_1 \dot{x} + a_0 x = 0$ ist dann stabil, wenn

alle Koeffizienten $a_i > 0$ UND
alle Hauptabschnittsdeterminanten (auch Hauptminoren genannt) positiv sind.

Insbesondere gilt für $a_2 \cdot \ddot{x} + a_1 \dot{x} + a_0 x = 0$ , dass die Hauptabschnittsdeterminanten von

$$\begin{pmatrix}a_1 & 0\\0 & a_0\end{pmatrix}$$ zu überprüfen sind. Das ist jedoch schon durch das erste Kriterium erfüllt.

Bei $a_3 \cdot x^{(3)} + a_2 \cdot \ddot{x} + a_1 \dot{x} + a_0 x = 0$ ist

$$\begin{pmatrix}a_2 & a_0 & 0\\1 & a_1 & 0\\ 0 & a_2 & 1\end{pmatrix}$$

zu überprüfen, also:

$a_2 > 0$ ?
$a_2 \cdot a_1 - a_0 > 0$ ? (the determinant of size 2 and 3 is the same)

Schaltungen

Subtrahierer: $U_A = \frac{R_0 (R_1 + R_3)}{R_1 (R_0 + R_2)} \cdot U_2 - \frac{R_3}{R_1} \cdot U_1$
Invertierender Addierer: $U_A = - \left (\sum_{i=1}^{N-1} \frac{U_i}{R_i} \right ) \cdot R_N$
Integrierer: $- \frac{1}{RC} \int U_E \mathrm{d}t$
Differenzierer: $U_A = - R_N \cdot I_E$
Invertierender OP: $y = - \frac{R_N}{R_1}$
Nicht-Invertierender OP: $y = \frac{R_N+R_1}{R_1}$
Parallelverfahren (vgl. ES 2 - 37)

Typische Klausur

Die Klausuren sind alle sehr ähnlich zu einander:

Regelung
- Zeitkonstante Regelung: 7 Punkte
  - Definition Regelung / Steuerung: 1 Punkt
  - Laplace-Bereich / Übergangsfunktion: 1 Punkt
  - Transformationstabelle / Differentialgleichung: 1 Punkt
  - Ist ein gegebenes System stabil? (mit Übergangsfunktion): 1 Punkt
  - PID-Regler / stabilität / Fehler: 2 Punkte
  - Gütegrad von Regelungssystem: 1.5 Punkte
  - Bode-Diagramm: 1.5 Punkte
- Zeitdiskrete Regelung: 8 Punkte
  - Aliasing / Abtasttheorem: 1 Punkt
  - Z-Transformierte Übertragungsfunktion G(z): 2 Punkte
  - Z-Transformierte / PD-Regelalgorithmus: 1 Punkt
  - Z-Transformierte / Regelkreis: 1 Punkt
  - Differenzialgleichungen → Differenzengleichung: 2 Punkte
  - Digital vs. Kontinuierlich (Diskretisierung): 1 Punkte
  - Stabilität eines Systems: 1 Punkt
Rechnerarchitekturen / Busse / Operationsverstärker
- Rechnerarchitekturen und Busse: 10 Punkte
  - Watchdog: 1 Punkt
  - Echtzeitausgabeeinheit (Konzept, Definition, Eigenschaften): 1 Punkt
  - Befehlsskalarer Prozessor / Sprungbedingungen: 1 Punkt
  - Bus-Eigenschaften: 2 Punkte
    - Echtzeitfähig oder nicht
    - Synchron ↔ asynchron
    - Getrennter Adress- und Datenbus ↔ multiplex
    - Burst-Datentransfer erlaubt oder auch nicht
- Operationsverstärker in Analog- und Digitaltechnik: 8 Punkte
  - Schaltsymbol Operationsverstärker (mit Versorgungsspannung), Anschlüsse beschriften: 1 Punkt
  - Eingangsströme (real/ideal): 1 Punkt
  - Operationsverstärker: 4 Punkte
  - Prinzip Operationswandler: 1 Punkt
  - Asymmetrische / differenzielle Datenübertragung: 1 Punkt
  - A/D-Wandler: 2 Punkte
Echtzeitkommunikation / Programmierung: 7 Punkte
- ISO / OSI-Schichtenmodell: 1.5 Punkte
- Manchester-Codierung: 1.5 Punkte
- Übertragungsfehler: 1.5 Punkte
- CAN-Dataframes: 2.5 Punkte
- Zusätzliche Forderungen an Echtzeitsysteme: 1 Punkt
- FPP-Scheduling: 2 Punkte
- Periodenabweichung: 2 Punkte
- Optimales Scheduling: 1 Punkt
- Schwankungen: 1 Punkt
Echtzeit-OS / SPS
- Echtzeit-OS: 7 Punkte
  - Zusätzliche Anforderungen: 1 Punkt
  - Zusätzliche Anforderungen Middleware: 1 Punkt
  - Beispiele: 1 Punkt
  - Sperrsynchronisation: 2 Punkte
  - Seitenadressierung: 2 Punkte
- SPS: 8 Punkte
  - FUP → AWL: 2.5 Punkte
  - FUP → Structured Text: 2.5 Punkte
  - 3 Verarbeitungsschirtte von SPS im zyklischen Programmbetrieb: 1 Punkt
  - Konventionelle SPS ↔ Soft: 1 Punkt
  - Graphische ↔ textuelle Programmiersprache: 1 Punkt

Material und Links

Vorlesungswebsite
Ilias
Klausur-Musterlösungen
Meine Anki-Karten (bestehend aus Teilen von Echtzeitsysteme - KIT Wörn und Echtzeitsysteme KIT Wörn Wissensfragen Klausur sowie weiteren Karten)

StackOverflow:

What is the difference between DMA and memory-mapped IO?

Übungsbetrieb

Wo sind die Übungsblätter: ? (Ilias?)
Abgabeform: Keine Abgabe
Abgabe (wochentag): Keine Abgabe
Rücknahme: -
Turnus: ?
Übungsschein verpflichtend: Es gibt keinen Übungsschein.
Bonus durch Übungsschein: Es gibt keinen Übungsschein.

Termine und Klausurablauf

Datum: Donnerstag, der 24.09.2015 von 14:00 Uhr-15:00 Uhr (Quelle)
Ort: Gehrtsen HS (Geb. 30.21)
Punkte: 60
Punkteverteilung: 4 Aufgaben à 15 Punkte:

Regelung
Rechnerarchitekturen, Busse, Operationsverstärker, Analog-/Digitaltechnik
Echtzeitkommunikation und Echtzeitprogrammierung
Echtzeitbetriebssysteme und SPS

Bestehensgrenze: ?
Übungsschein: Gibt es nicht.
Bonuspunkte: Gibt es nicht.
Ergebnisse: Die Noten der Klausur werden am Schwarzen Brett im IAR-IPR (Geb. 40.28, Foyer, links) ausgehängt. Noch ist nichts da (Stand: 24.09.2015)
Einsicht: wird über Ilias bekannt gegeben
Erlaubte Hilfsmittel: Keine

Fazit

Dieses Modul kann man sich getrost schenken. Wenn man vorher nichts von Regelungstechnik weiß, ist man hinterher auch nicht schlauer. Es scheint so zu sein, dass die klausurrelevanten Teile alle in den Übungen besprochen werden.

Die bereitgestellten Materialien hätten besser sein können. Es gibt zwar ein Skript welches sogar eine ISBN-Nummer hat, aber insbesondere bei dem Regelungstechnik-Teil ist es nicht sonderlich hilfreich. Ich hatte das Gefühl, dass mir da einfach Grundlagen fehlen. Diese werden auch nicht erklärt. Zu den PDF-Folien muss man sagen, dass diese zwar schnell im Ilias hochgeladen wurden, aber teilweise zu viele Informationen hatten. Es war nicht klar, was wichtig ist. Außerdem hat teilweise, wenn man nur die Folien angesehen hat, der Kontext gefehlt.

Ein Highlight waren Videos, in denen Roboter Kugeln auf einem Tablett sehr schnell transportieren. Dazu müssen die Roboter das Tablett im richtigen Winkel neigen. Das war ein Highlight der Vorlesung. Schade, dass nie erklärt wurde wie so etwas berechnet wird.

Vorwissen in der Regelungstechnik und bei Differentialgleichungen ist sicher hilfreich.

Die Klausurvorbereitung besteht hauptsächlich aus Auswendiglernen. Verständnis ist wohl nicht nötig (mal schauen… ich konnte keinen Notenschlüssel finden.), da die Klausuren immer nahezu identisch aufgebaut sind.

Meine Empfehlung an Studenten ist, in die Übung zu gehen. Da kommen die klausurrelevanten Sachen. Teilweise 1-zu-1 die gleichen Fragen.

Verbesserungsvorschläge für die Dozenten hätte ich auch ein paar:

Es sollte Tutorien geben.
Es sollte einen verpflichtenden Übungsschein geben. Mit 50% der Punkte hat man den Übungsschein, ab 75% gibts Bonuspunkte für die Klausur. Diese Übungsblätter werden von den Tutoren korrigiert und besprochen. Wenn die Studenten es reihenweise nicht schaffen merkt man schon, wo man die Vorlesung verbessern muss.
Veröffentlichung der Musterlösungen zu den Klausuren.
Anpassung des Skripts, z.B. wurde der VME-Bus durch den PCIe-Bus ersetzt.
Anpassung des Modulhandbuchs. Man sollte den Leuten empfehlen, zuvor die Vorlesung “Betriebssysteme” gehört zu haben.
Ein Wikibook beginnen. Wenn man das für jedes Modul machen würde, könnte man auch Abhängigkeiten besser sehen und Studenten könnten fehlendes Wissen leichter nachholen. Außerdem würde man das Wissen frei verfügbar machen und man könnte die Exzellenz der Lehre zeigen, falls man wirklich der Meinung ist die Lehre am KIT sei exzellent. Über die Diskussionsseiten könnten Studenten und andere weitgehend anonym ihre Fragen stellen und Verbesserungsvorschläge machen. Über das Wiki-System könnten freiwillige Helfer insbesondere bei zeitaufwendigen Grafiken helfen.

Markovsche Ketten - Klausur

info@martin-thoma.de (Martin Thoma) — Mon, 13 Apr 2015 13:42:00 +0200

Dieser Artikel beschäftigt sich mit der Vorlesung „Markovsche Ketten“ am KIT. Er dient als Prüfungsvorbereitung. Ich habe die Vorlesungen bei Herrn Dr. Bernhard Klar im Sommersemester 2015 gehört. Der Artikel wird bis zur Klausur laufend erweitert.

Behandelter Stoff

Es wäre toll, wenn ich von jeder Vorlesung einen Mitschrieb hochladen könnte. Gibt es Leute, die eine Kamera / einen Scanner haben und mir ihren Mitschrieb als JPG-Bilder schicken würden? Einfach an info@martin-thoma.de schicken.

Vorlesung

Zur Vorlesung gibt es das Skript “Markov-Ketten” von Frau Prof. Dr. Bäuerle. (Ich habe eine Version von 2012).

Datum	Kapitel	Inhalt
13.04.2015	0. Beispiele (Mitschrieb)	Einführung in Markovketten mit vielen Beispielen (Weg des Betrunkenen, Ehrenfest-Modell, Irrfahrt, Vererbung); absorbierende Zustände; stochastische Matrix
16.04.2015	1.	Konstruktion von Markov-Ketten
20.04.2015
21.04.2015	3.10 - 4.6 (Mitschrieb)	Total-Variationsabstand, `$d(\mu, \nu) = \frac{1}{2} \sum_{i \in S} \|\mu(i)- \nu(i)\|$`, Periode, aperiodisch, ein Konvergenzsatz, Kopplungsargument

Material und Links

KIT

Sonstiges

Markov Chains: A very short introduction to markov chains with beautiful visualizations

Wichtigster Stoff

Wie immer Definitionen (Markovkette, transient, rekurrent, Klasse, irreduzibel)

Wenn man die Matrix so umsortiert, dass rechts unten die rekurrenten Zustände sind (das ist nicht immer eine Einheitsmatrix!), Dann heißt die Matrix links oben $Q$ und rechts oben $R$ . Dann gilt:

$(E-Q)^{-1} \cdot R$ : Absorptionszeit
$(E-Q)^{-1} \cdot \begin{pmatrix}1\\\vdots\\1\end{pmatrix}$ : Schritte bis zur Absorption.
Invariantes Maß finden: $\pi P = \pi$ bzw. $\pi Q = 0$ im zeitkontinuierlichen Fall

Übungsbetrieb

Es gibt Dienstags und Mittwochs Tutorien.

Wo sind die Übungsblätter: Ilias
Abgabeform: Handgeschrieben (?)
Abgabe: ?
Rücknahme: ?
Turnus: Wöchentlich
Übungsschein verpflichtend: Nein
Bonus durch Übungsschein: Nein

Termine und Klausurablauf

Datum: Montag, der 03.08.2015 von 11:00 bis 13:00 Uhr (Quelle)
Ort: Daimler-Hörsaal (Geb. 10.11, Quelle)
Punkte: 60
Punkteverteilung: 6 Aufgaben mit 9-13 Punkten
Bestehensgrenze: ?
Übungsschein: ?
Bonuspunkte: ?
Ergebnisse: ?
Einsicht: Montag, 19.10.2015, 13:00 Uhr - 13:30 Uhr, Im Raum 2.071, Mathematikgebäude
Erlaubte Hilfsmittel: ?

Wahrscheinlichkeitstheorie-Klausur

info@martin-thoma.de (Martin Thoma) — Mon, 13 Apr 2015 10:37:00 +0200

Dieser Artikel beschäftigt sich mit der Vorlesung „Wahrscheinlichkeitstheorie“ am KIT. Er dient als Prüfungsvorbereitung. Ich habe die Vorlesungen bei Herrn Prof. Dr. Henze im Sommersemester 2015 gehört. Der Artikel wird bis zur Klausur laufend erweitert.

Der Dozent veweist immer wieder auf die Vorlesung Analysis III und insbesondere das Skript von Prof. Hundertmark. Kann mir das jemand schicken?

Behandelter Stoff

Vorlesung

Datum	Kapitel	Inhalt
13.04.2015	Maßtheoretische Grundlagen 0.1 - 1.13	Wiederholung von Begriffen: Maß, `$\sigma$`-Algebra, Halbring, Dynkin-System, Durchschnittsstabilität (`$A, B \in \mathcal H \Rightarrow A \cap B \in \mathcal H$`)
15.04.2015	?	?
20.04.2015	?	?
27.04.2015	?	?
29.04.2015	?	?
04.05.2015	?	?
11.05.2015	?	?
13.05.2015	?	?
18.05.2015	?	?
27.05.2015	?	?
01.06.2015	?	?
03.06.2015	?	?
10.06.2015	?	?
15.06.2015	?	?
22.06.2015	?	?
24.06.2015	?	?
29.06.2015	?	?
06.07.2015	?	?
08.07.2015	?	?
13.07.2015	?	?

Material und Links

Übungsbetrieb

Wo sind die Übungsblätter: ?
Abgabeform: ?
Abgabe: ?
Rücknahme: ?
Turnus: ?
Übungsschein verpflichtend: ?
Bonus durch Übungsschein: ?

Termine und Klausurablauf

Datum: Donnerstag, der 30.07.2015 von 11:00 bis 13:00 Uhr (Quelle)
Ort: Hertz-Hörsaal (Geb. 10.11, Quelle)
Punkte: ?
Punkteverteilung: ?
Bestehensgrenze: ?
Übungsschein: ?
Bonuspunkte: ?
Ergebnisse: ?
Einsicht: ?
Erlaubte Hilfsmittel: ?

Python Code Documentation

info@martin-thoma.de (Martin Thoma) — Wed, 08 Apr 2015 19:39:00 +0200

Documentating your code is important when you make non-trivial projects. The standard way to document Python code is with Sphinx. You write the documentation files with reStructuredText.

One of the most important Sphinx plugins is autodoc. This allows you to generate the documentation of modules, classes and functions automatically by using their docstrings.

There are 3 standard ways to write docstrings: Sphinxy, Googley or NumPyDocy.

The Sphinx Way

def preprocessing(self, algorithms):
    """Apply preprocessing algorithms.

    :param algorithms: Preprocessing allgorithms which get applied in order.
    :type algorithms: a list objects

    >>> import preprocessing
    >>> a = HandwrittenData(...)
    >>> preprocessing_queue = [(preprocessing.scale_and_shift, []),
    ...                        (preprocessing.connect_strokes, []),
    ...                        (preprocessing.douglas_peucker,
    ...                         {'EPSILON': 0.2}),
    ...                        (preprocessing.space_evenly,
    ...                         {'number': 100,
    ...                          'KIND': 'cubic'})]
    >>> a.preprocessing(preprocessing_queue)
    """
    assert type(algorithms) is list
    for algorithm in algorithms:
        algorithm(self)

The Google Way

Google documented its format at http://google-styleguide.googlecode.com/. (I’m wondering where that will move, now that they plan to shut code.google.com down. Probably to GitHub.)

def preprocessing(self, algorithms):
    """Apply preprocessing algorithms.

    Args:
      algorithms (a list objects): Preprocessing allgorithms which get
        applied in order.

    Examples:

      >>> import preprocessing
      >>> a = HandwrittenData(...)
      >>> preprocessing_queue = [(preprocessing.scale_and_shift, []),
      ...                        (preprocessing.connect_strokes, []),
      ...                        (preprocessing.douglas_peucker,
      ...                         {'EPSILON': 0.2}),
      ...                        (preprocessing.space_evenly,
      ...                         {'number': 100,
      ...                          'KIND': 'cubic'})]
      >>> a.preprocessing(preprocessing_queue)
    """
    assert type(algorithms) is list
    for algorithm in algorithms:
        algorithm(self)

To get the google way render well in Sphinx, you need Napoleon.

The NumPyDoc Way

def preprocessing(self, algorithms):
    """Apply preprocessing algorithms.

    Parameters
    ----------
    algorithms : a list objects
        Preprocessing allgorithms which get applied in order.

    Examples
    --------
    >>> import preprocessing
    >>> a = HandwrittenData(...)
    >>> preprocessing_queue = [(preprocessing.scale_and_shift, []),
    ...                        (preprocessing.connect_strokes, []),
    ...                        (preprocessing.douglas_peucker,
    ...                         {'EPSILON': 0.2}),
    ...                        (preprocessing.space_evenly,
    ...                         {'number': 100,
    ...                          'KIND': 'cubic'})]
    >>> a.preprocessing(preprocessing_queue)
    """
    assert type(algorithms) is list
    for algorithm in algorithms:
        algorithm(self)

To make numpydoc render well, you need the numpydoc sphinx extension.

Python ctypes

info@martin-thoma.de (Martin Thoma) — Thu, 02 Apr 2015 22:06:00 +0200

One pseudo-problem people often mention when talking about Python is that Python is (too) slow. What they seem to forget or don’t know is that you can call C code from Python with ctypes. So you can get almost as fast as you can get with C; you’re not limited by the language in that respect. And most of the time your code has other issues when it is too slow.

Now, you can also wrap Rust code with ctypes for Python ☺

Motivation

As a very simple example, I prepared a dumb Fibonacci implementation written in Python:

def fib(n):
    """Calculate the n-th Fibonacci number."""
    if n <= 1:
        return n
    else:
        return fib(n-1) + fib(n-2)

if __name__ == '__main__':
    n = 37
    print("The %ith Fibonacci number is %i." % (n, fib(n)))

and exactly the same for Rust

pub extern fn fib(n: u32) -> u32 {
    if n <= 1 {
        return n;
    } else {
        return fib(n-1) + fib(n-2);
    }
}

fn main() {
    let n = 37;
    println!("The {0}th Fibonacci number is {1}.", n, fib(n))
}

The execution times are quite different. Rust needs 0.40 seconds while Python 3 needs 15.7 seconds. That is almost 40× the time of Rust!

Wouldn’t it be great if we could call the Rust function from Python?

Example

I’ll explain in the next chapters what is done, but at first you should see that there are only minor changes / overhead:

fibonacci.rt:

#![crate_type = "dylib"]

#[no_mangle]
pub extern fn fib(n: u32) -> u32 {
    if n <= 1 {
        return n;
    } else {
        return fib(n-1) + fib(n-2);
    }
}

Call rustc -O fibonacci.rt to generate the library.

Python:

#!/usr/bin/env python

import ctypes

fiblib = ctypes.CDLL("./libfibonacci.so")
fib = fiblib.fib
n = 37
print("The %ith Fibonacci number is %i." % (n, fib(n)))

Now, taking the Python code, it takes only 0.44 seconds!

What happens

The line #![crate_type = "dylib"] tells rustc that it has to create a dynamic library.

The line #[no_mangle] tells the compiler not to mangle the name fib. This is important so that we can later use it from Python. (I think names are mangled to prevent name clashes … so it’s a kind of name-spacing.)

Then we load the C DLL with ctypes.CDLL("./libfibonacci.so") and use it as expected. Pretty easy, isn’t it?

Caveats

Python makes some things very simple which are not that simple at all. Think about numbers, for example. In Rust, you have integers with 64 bits. But in Python you have arbitrary length integers. This might lead to problems.

The Rust Programming Language

info@martin-thoma.de (Martin Thoma) — Thu, 26 Mar 2015 16:25:00 +0100

Rust is a compiled programming language which aims to replace C++. The designers wanted it to be similar fast, give the programmer similar fine-grained control over memory management, but more safety.

Rust does so by introducing the concept of ownership, borrowing and lifetimes of variables. Rust also makes sure that after the scope is finished, the memory of variables gets de-allocated.

I think FFI is interesting. It allows Rust to “talk” with C code.

Rust 1.0 was just released (source). This means the language should be stable.

If you want to know more, you should take a look at the beautiful A 30-minute Introduction to Rust.

Syntax Example

fn fib(n: u32) -> u32 {
    if n <= 1 {
        return n;
    } else {
        return fib(n-1) + fib(n-2);
    }
}

fn main() {
    let n = 36;
    println!("The {0}th Fibonacci number is {1}.", n, fib(n))
}

Video introduction

Comparison

benchmarksgame shows that rust is similar fast as C++. For some games it is faster, for some slower. Sometimes it needs less code, sometimes more.
stackoverflow has only 1781 questions of which are 102 unanswered (5.7%). C++ has 356,824 questions and 55,172 questions are not answered (15.5%). But I guess that will change as soon as more people use Rust (and libraries written in Rust).
GitHub: Wow, Rust seems to be growing fast!

Tools

As rust is still in very heavy development, you should build it yourself from the source at GitHub. Don’t worry, there are very detailed instructions. (But it takes a lot of time to compile.)

cargo package manager
Syntax Highlighting and auto completion for ST3 (via Package Control)
- Auto completion with Racer

Comparison to C++

1: No crashes because of code misuse (as long as you do not use unsafe {} )

2: No memory leaks

3: No data races (so you can thread this thing like mad, but better is has async task handling (crs pattern to). So lighting up your machines cores is no prob.

4: Cross platform (really much better than most)

5: Inbuilt version management, build system and test harness (with benchmarking to).

6: Package management system (no more I Cannot build XX, it becomes automatic)

7: Inbuilt generics and traits (c++ concepts and more)

8: Very strongly typed (near zero runtime)

9: Very fast

10 compiles into a c lib basically (easy integration)

Source: dirvine

Conclusion

Rust looks very, very interesting. I am usually not a fan of new languages (e.g. I don’t see a reason to use Go, Swift … not to speak of Clojure, Julia, TypeScript, Dart). Most of the time, I don’t see anything new which is better than in existing languages and almost always the material around (tools, libraries, community) is much worse than in existing solutions.

This seems to be different with Rust. Rust seems to provide everything I enjoy from Python, but in the fast, explicitly typed world.

@Johannes: Thanks for pointing me to Rust ☺

Dreamspark SDM Odysee

info@martin-thoma.de (Martin Thoma) — Tue, 24 Mar 2015 13:04:00 +0100

I need to have Windows for work. As a student I should have free access to it. But when I try to download it, I have to use Microsofts “Secure Download Manager” (SDM). As a Ubuntu (Linux) user, this is easier said than done.

TL;DR

You need a Windows PC to download Windows. Ask a friend to download it. That might be the easiest way. If you have a Windows PC and get the same error as I get, you might need to re-install the system. (No, rebooting does not help.)

What I’ve tried and what does not work:

Wine
VMs
Avoiding SDM

What I am currently trying: Installing Windows 7 on another computer.argh

UPDATE: How to bypass Secure Download Manager while downloading from Dreamspark looks very promising.

The problem

Using the Windows 8 systems at my university gives:

Error message when trying to install SDM

Das Feature, das Sie verwenden möchten, befindet sich auf einer Netzressource, die nicht zur Verfügung steht.

Klicken Sie auf “OK”, um den Vorgang zu wiederholen. Oder geben Sie in das untenstehende Feld den Pfad zu einem anderen Ordner ein, der das Installationspaket “SDM_DE.msi” enthält.

(Sorry, I only get the German error message.)

When I give the path of the SDM_DE.msi I get:

Die Datei “Z:\sdm\SDM_DE.msi” ist kein gültiges Installationspaket für das Produkt “Secure Download Manager”. Suchen Sie das Installationspaket “SDM_DE.msi” in einem Order, von dem aus Sie “Secure Download Manager” installieren können.

Then I got desperate and followed threadsofscience.wordpress.com/2013/02/13/downloading-dreamspark-microsoft-windows-on-ubuntu (similar: http://boris-spinner.de/secure-download-manager-sdm-unter-linux-ausfuehren/).

It seems as if I need Linux to emulate Windows to get Windows … it’s a strange world.

wine: Bad EXE format for Z:\home\moose\Downloads\SDM_EN.msi.

Seems to be a 32-bit / 64-bit problem …

http://wiki.winehq.org/FAQ#32_bit_wineprefix
https://appdb.winehq.org/objectManager.php?sClass=version&iId=31542

After trying to fix it, I get

err:msidb:get_tablecolumns column 1 out of range err:msidb:get_tablecolumns column 2 out of range fixme:storage:create_storagefile Storage share mode not implemented. err:msidb:get_tablecolumns column 1 out of range err:msidb:get_tablecolumns column 2 out of range err:msidb:get_tablecolumns column 1 out of range err:msidb:get_tablecolumns column 2 out of range err:msidb:get_tablecolumns column 1 out of range err:msidb:get_tablecolumns column 2 out of range err:msidb:get_tablecolumns column 3 out of range err:msidb:get_tablecolumns column 1 out of range err:msidb:get_tablecolumns column 2 out of range err:msidb:get_tablecolumns column 3 out of range err:msidb:get_tablecolumns column 1 out of range err:msidb:get_tablecolumns column 2 out of range err:msidb:get_tablecolumns column 3 out of range err:msidb:get_tablecolumns column 1 out of range err:msidb:get_tablecolumns column 2 out of range err:msidb:get_tablecolumns column 3 out of range

Reverse Engineering … a little bit

Taking a look at the 1234567890ab.sdx file reveals that it contains only a single url:

http://kit.onthehub.com/WebStore/Account/SdmAuthorize.aspx?o=12345678-1234-1234-123a-12234567890a&ws=12345678-1234-1234-1234-1234567890ab&uid=12345678-1234-1234-1234-1234567890ab&abc=5

(I’ve replaced the numbers by 123… to avoid problems with other people using my link to download stuff)

However, I cannot simply download it there. There is a page with two downloads, but when I click on download nothing happens.

SDM download page

So I use Chrome developer tools (Ctrl+shift+i) to see the request.

It’s a GET request which gets the response:

<information>
    <oiopua>12345678-1234-1234-1234-1234567890ab</oiopua>
    <edv>1234567890^^1234567890abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789</edv>
    <linkAvailable>1</linkAvailable>
    <errorTextKey />
    <invokeExternalDownload>0</invokeExternalDownload>
    <fileURL>http://software.dreamspark.com/dreamspark/GERMAN/de_windows_8.1_with_update_x64_dvd_4048209.01.sdc</fileURL>
</information>

Although I can download the .sdc file, I cannot use it. It is an encrypted file (which would give the .iso, but I don’t know how to encrypt it.).

If you are interested in the file format, you might want to read Secure Digital Container

VMs

Another try is installing a VM. You can download some at https://www.modern.ie/en-us/virtualization-tools#downloads

I downloaded IE11, Win 8.1. Importing the VM took about 15 minutes on my computer. Then I got another error:

VM import error

Installing Windows 7

12:50 I fortunately had a Windows 7 DVD lying around. Installing it will probably take another 2 hours or so…

13:23 - argh … I just found another reason to hate Windows. I’ve just installed Windows 7. Cable-based network does not work out of the box. I have to install drivers from http://www.helpjet.net/files-Acer-TravelMate-5735Z.html#ANetwork

HTTPie

info@martin-thoma.de (Martin Thoma) — Sun, 22 Mar 2015 17:50:00 +0100

Johannes recently showed me httpie. It’s a very nice tool for sending HTTP requests. It is much simpler to use than curl.

Installation

$ sudo -H pip install httpie

Usage

$ http --form POST localhost:5000/api/recording recording=[{}] id=1
HTTP/1.0 200 OK
Content-Length: 21
Content-Type: text/html; charset=utf-8
Date: Sun, 22 Mar 2015 15:50:27 GMT
Server: Werkzeug/0.10.1 Python/2.7.8

edit recording with 1

The output is colored in a very nice way:

HTTPie in action

Python and Encodings

info@martin-thoma.de (Martin Thoma) — Sat, 14 Feb 2015 14:28:00 +0100

Working with encodings different from ASCII or UTF-8 has always been work which I don’t like. It doesn’t feel very constructive to just make Python read a file / print some output.

In the following, I will describe some strategies which might help you.

Test script

Copy the following text to a text file test.txt:

Die süße, kleine, lärmende Überfliegerin lebt in der Haute-Côte-Nord.
Dort hat es momemtan 32°C.

On Debian based systems you will get the information which type of encoding it has like this:

$ file test.txt
test.txt: UTF-8 Unicode text

This is probably the best result. But to make sure that we know how to deal with other encodings, you can change the encoding like this:

$ iconv -f UTF-8 -t ISO-8859-1 test.txt > test-iso-8859-1.txt
$ file test-iso-8859-1.txt 
test-iso-8859-1.txt: ISO-8859 text

Source code encoding

A first important step is to define the source code encoding. This is done with a comment. The first lines of Python code should probably always look like this:

#!/usr/bin/env python
# -*- coding: UTF-8 -*-

PEP-0263 explains it.

Printing encoding problems

The following error occurs when you try to print non-UTF-8 stuff with Python via Sublime Text:

[Decode error - output not utf-8]

The same code, executed via ZSH, gives:

Die s��e, kleine, l�rmende �berfliegerin lebt in der Haute-C�te-Nord.
Dort hat es momemtan 32�C.

You can fix that by adjusting the code the following way:

#!/usr/bin/env python
# -*- coding: UTF-8 -*-

# Make it work with Python 2 and Python 3
import sys
PY3 = sys.version > '3'

if not PY3:
    from future.builtins import open

# Specify the encoding while opening it
with open("test-iso-8859-1.txt", encoding="ISO-8859-1") as f:
    content = f.read()
content = content.encode('UTF-8', 'replace')
print(content)

The two important points are

Specifying the encoding while opening the file
Encode the content with UTF-8

These three little steps helped me to deal with non-UTF-8 encodings.

Reading and Writing Files with Python

info@martin-thoma.de (Martin Thoma) — Thu, 12 Feb 2015 22:11:00 +0100

Reading (and writing) files with Python is very easy. Here are some minimalistic code examples for beginners.

Reading

A very common way to handle the contents of a file is by reading the file completely and then working with a single big string. The only reason not to do so is because your file is too big. For your information, this is how long reading a file takes (all times are in seconds, reading is executed 100 times):

file size         min       max    median   average
---------------------------------------------------
KiB         0.0000    0.0008    0.0000    0.0002
KiB        0.0000    0.0009    0.0001    0.0002
MiB          0.0003    0.0013    0.0005    0.0006
MiB         0.0041    0.0079    0.0047    0.0049
MiB        0.0627    0.0778    0.0655    0.0663

As you can see, reading files is quite fast. However, I would recommend the start thinking if reading the complete file is appropriate when the file size exceeds 100 MiB as you might get different problems then. For example, the content of the file might not fit in your main memory (typically 4GiB, type cat /proc/meminfo | grep MemTotal to get how much main memory your computer has).

See source code for details how I measured it.

Completely

with open('filename.txt') as f:
    content = f.read()

is all you need to read a text file completely in a Python string variable content. You could, for example, split the whole string to a list of lines:

lines = content.split('\n')

Line by line

You can read a text file line by line, but keep in mind that this will still have the endline character \n in each line!

with open('filename.txt') as f:
    for line in f:
        print(line)

Writing

To open files, you have to specify a mode. This is either reading, writing or appending (binary or text data). If you don’t specify it, the default value is reading. So for writing you have to specify it:

with open('filename.txt', 'w') as f:
    f.write("Hello world!")

Python 3

Python 3 has some new features which are very nice to have. To get them in Python 2, you have to add

from __future__ import print_function

at the beginning of your code before other imports happen. Now you can print to files:

from __future__ import print_function

with open('file.txt', 'w') as f:
    print("Hello World!", file=f)

The print function has also a “end” parameter which defaults to \n. This means it adds \n automatically at the end of each printed line. You might not want that e.g. in a Windows environment where it should be \r\n.

with

You might wonder what with does. My advice for newbees would be not to worry too much about it, it is just the way you juse I/O with Python. If you come from the C / C++ world, you might know that you have to close files when you opened them. The with statement makes sure that the file is closed when the block is finished.

Python and CSV

info@martin-thoma.de (Martin Thoma) — Sun, 08 Feb 2015 12:48:00 +0100

Python has a very nice module called csv which makes working with CSV very easy. This mini article is only a reminder for me so that I can easily find how to use it when I forget once again how it is used exactly.

Reading CSV files

try:
    from future.builtins import open
except:
    pass

import csv

with open('eggs.csv', 'rt', newline='') as csvfile:
    csvreader = csv.reader(csvfile, delimiter=';', quotechar='"')
    next(csvreader, None)  # skip the headers
    for row in csvreader:
        print(', '.join(row))

Writing CSV files

try:
    from future.builtins import open
except:
    pass

import csv

with open('eggs.csv', 'w', newline='') as csvfile:
    csvwriter = csv.writer(csvfile,
                           delimiter=';',
                           quotechar='"',
                           quoting=csv.QUOTE_MINIMAL)
    csvwriter.writerow(['Spam'] * 5 + ['Baked Beans'])  # Write header
    for row in container:
        csvwriter.writerow(row)  # Write data

Python Shell autocomplete

info@martin-thoma.de (Martin Thoma) — Thu, 05 Feb 2015 09:58:00 +0100

One feature I really miss in Pythons interactive shell is tab autocompletion. Thanks to blog.e-shell.org I know how to get it:

>>> import rlcompleter, readline
>>> readline.parse_and_bind('tab:complete')
>>> smtplib.
smtplib.CRLF                      smtplib.__getattribute__(
smtplib.LMTP                      smtplib.__hash__(
smtplib.LMTP_PORT                 smtplib.__init__(
smtplib.OLDSTYLE_AUTH             smtplib.__name__
smtplib.SMTP                      smtplib.__new__(
smtplib.SMTPAuthenticationError(  smtplib.__package__
smtplib.SMTPConnectError(         smtplib.__reduce__(
smtplib.SMTPDataError(            smtplib.__reduce_ex__(
smtplib.SMTPException(            smtplib.__repr__(
smtplib.SMTPHeloError(            smtplib.__setattr__(
smtplib.SMTPRecipientsRefused(    smtplib.__sizeof__(
smtplib.SMTPResponseException(    smtplib.__str__(
smtplib.SMTPSenderRefused(        smtplib.__subclasshook__(
smtplib.SMTPServerDisconnected(   smtplib._addr_only(
smtplib.SMTP_PORT                 smtplib._have_ssl
smtplib.SMTP_SSL                  smtplib.base64
smtplib.SMTP_SSL_PORT             smtplib.email
smtplib.SSLFakeFile               smtplib.encode_base64(
smtplib.__all__                   smtplib.hmac
smtplib.__class__(                smtplib.quoteaddr(
smtplib.__delattr__(              smtplib.quotedata(
smtplib.__dict__                  smtplib.re
smtplib.__doc__                   smtplib.socket
smtplib.__file__                  smtplib.ssl
smtplib.__format__(               smtplib.stderr

Now if you want to always have that, you can get it by editing your ~/.pythonrc:

import rlcompleter, readline
readline.parse_and_bind('tab:complete')

You also have to make sure that the environment variable PYTHONSTARTUP is set to ~/.pythonrc. In my case, I had to add the line export PYTHONSTARTUP=~/.pythonrc to ~/.zshrc.

I have also seen that there is another, “more powerful”, Python shell called ipython. However, I don’t understand how this shell supports autocompletion (see How does ipython3 import autocomplete work?).

Execute Python on Apache2

info@martin-thoma.de (Martin Thoma) — Sun, 01 Feb 2015 15:11:00 +0100

I have to run a very simple Python script on an Apache2 web server. These were the steps with which I made it work:

Find the Apache2 config

$ /usr/sbin/apache2 -V

It is in /etc/apache2/apache2.conf

Add CGI

Somewhere in the /etc/apache2/apache2.conf there is a <Directory ...> part. I’ve added Options ExecCGI and AddHandler cgi-script .py. Now this part looks like this:

<Directory /var/www/>
    Options Indexes FollowSymLinks ExecCGI
    AddHandler cgi-script .py
    AllowOverride None
    Require all granted
</Directory>

Test it

Create the following test.py file:

#!/usr/bin/env python
# -*- coding: UTF-8 -*-

print("Content-Type: text/html\n")
print("Hello World! The answer to live, the universe and everything is %i." %
      (2*21))

Now call http://localhost/test.py.

If that doesn’t work, take a look at the apache log files:

$ tail -f /var/log/apache2/error.log

GUI programming with Python

info@martin-thoma.de (Martin Thoma) — Fri, 23 Jan 2015 14:39:00 +0100

A graphical user interface (GUI) is essential for applications which should be used by standard computer users (non-developers, not computer scientists, …). However, I have almost no experience with GUI development outside of the web.

Multiple GUI toolkits exist and the only one I have ever used is Tk for a very, very simple GUI.

In this article, I want to share some of my thoughts about GUI development with Python as a beginner. I might update this in future.

GUI toolkits

A widget toolkit, widget library, GUI toolkit, or UX library is a library or a collection of libraries containing a set of graphical control elements (called widgets) used to construct the graphical user interface (GUI) of programs.

Source: Widget toolkit

Comparison

I am only interested in GUI toolkits

which work on Ubuntu 12.04+,
which have a Python binding,
which are OpenSource and have a good license,
which are used by others (and hence have enough documentation and examples)

It seems to me that only the following four toolkits fulfill these requirements: GTK, Qt, Tk, wxWidgets. Some of these have multiple Python bindings and all have multiple GUI builders / designers. I tried to find the most commonly used one. Please let me know if I should add something, replace something:

Name	GTK+	Qt	Tk	wxWidgets
Latest version (23.01.2015)	3.14.1	5.4.0	8.6.3	3.0.2
Official Website	gtk.org	qt-project.org	tcl.tk	wxwidgets.org
Initial release	1998	1995	1991	1992
Written in	C	C++	C	C++
Documentation	gtk.org/documentation.php	doc.qt.io	tkdocs.com	wxwidgets.org/docs
Tutorial	developer.gnome.org/gtk-tutorial/stable	qt-project.org/doc/qt-4.8/tutorials.html	tkdocs.com/tutorial	wxwidgets.org/docs/tutorials
StackOverflow questions	4,715	37,626	929	1,918
StackOverflow unanswered	1,159	9,704	208	429
License	LGPL 2.1	LGLP 3.0 (mutliple	BSD-style	wxWindows License
Python binding	PyGTK (docs)	PySide (docs), PyQt	Tkinter (docs)	wxPython (docs)
Python 3 support	yes	yes	yes?	yes
Designer	Glade Interface Designer	QtDesigner, QtCreator, QDevelop, Edyuk	SpecTcl	wxGlade
Famous applications	Gnome applications like Inkscape, OTR-Verwaltung	vlc, Virtual Box, KDE applications like K3B, Anki	I could not find any	Code::Blocks FileZilla 0 A.D.

There are also some applications which use custom UI toolkits:

Distribution of Random Variables when max gets applied

info@martin-thoma.de (Martin Thoma) — Thu, 22 Jan 2015 11:07:00 +0100

I just wanted to solve an exercise where I had random variables $X_1, \dots, X_n$ which were all $U([0, 1])$ distributed and $Y_n = \max(X_1, \dots, X_n)$ .

I wondered what the distribution of $Y_n$ is (for big $n$ ), so I wanted to plot it. How do I plot it? With Python, of course ☺

Here is the program:

#!/usr/bin/env python

import matplotlib.pyplot as plt
import numpy.random


def main():
    # Generate Data
    n = 10000
    numbers_a = numpy.random.uniform(size=samples)
    numbers_b = numpy.random.uniform(size=samples)
    numbers_c = numpy.random.uniform(size=samples)
    numbers_max = [max(a, b, c)
                   for a, b, c in zip(numbers_a, numbers_b, numbers_c)]

    # Plot data
    plt.hist(numbers_max)
    plt.title("Histogram")
    plt.xlabel("value")
    plt.ylabel("count")
    plt.show()


if __name__ == '__main__':
    main()

and here is the plot for $n = 2$

Plot of the maximum of 2 uniformly distributed variables with 10000 samples

If you increase to $n = 3$ you get:

Plot of the maximum of 3 uniformly distributed variables with 10000 samples

Improved version

I’ve also created an improved version which makes nicer plots, but might be harder to read:

#!/usr/bin/env python

import matplotlib.pyplot as plt
import numpy.random


def main(samples, n):
    # Generate Data
    numbers_l = []
    n = n + 1
    for _ in range(n):
        numbers_l.append(numpy.random.uniform(size=samples))

    # Plot data
    plots = []
    for i in range(1, n):
        # Build data structure
        sublist = numbers_l[:i]
        max_list = numbers_l[0]
        for numbers_list in sublist:
            for i, el in enumerate(numbers_list):
                max_list[i] = max(max_list[i], el)
        plot_i = plt.hist(max_list, histtype='step', bins=200)
        plots.append(plot_i)
    plt.title("Histogram")
    plt.xlabel("value")
    plt.ylabel("count")
    plt.show()


def get_parser():
    from argparse import ArgumentParser, ArgumentDefaultsHelpFormatter
    parser = ArgumentParser(description=__doc__,
                            formatter_class=ArgumentDefaultsHelpFormatter)
    parser.add_argument("-n",
                        dest="n",
                        default=3,
                        type=int,
                        help="n")
    parser.add_argument("-s", "--samples",
                        dest="samples",
                        default=1000,
                        type=int,
                        help="number of samples per Y_n")
    return parser


if __name__ == '__main__':
    args = get_parser().parse_args()
    main(args.samples, args.n)

When you call this script with

$ ./plot-random-variable.py -n 4 --samples 100000

it gives

Plot of the maximum of 1 - 4 uniformly distributed variables with 10000 samples

Bug Reporting - A users perspective

info@martin-thoma.de (Martin Thoma) — Wed, 21 Jan 2015 14:54:00 +0100

Bug reporting is extremely important. It helps developers to get aware of problems and hence get the possibility to do something against it. It is impossible to guarantee for any real, non-trivial software that it has no bugs. Even when you formally prove that it is correct, the prove might be wrong. However, when users report bugs one can get confident that the remaining bugs are appearing very rarely or causing not so much harm.

Reporting bugs is also important from a User Experience point of view. For me, it already helps to be able to report a bug / see that a bug was already reported. However, this is a pain in the ass for most software from a users perspective.

I’ve just had that experience for Caja (the file explorer of MATE).

Caja example

Get information

Caja crashed. I just wanted to create a new text file. Then it froze for about two seconds and closed. After another one or two seconds I got a crash report window. (I forgot to make a screenshot of that, but it looks like in the Google Chrome example)

I clicked on something like “examine locally” - whatever that means. This is the first point I have to critize. The user should always know or at least be able to get the information what such messages mean.

I was curious, so I clicked on it. Then this appeared:

Apport 'examine locally' window

This will launch apport-retrace in a terminal window to examine the crash.

That is fine.

Run gdb session

Run gdb session without downloading debug symbols

Update /var/crash/_usr_bin_caja.1000.crash with fully symbolic stack trace

That is not ok. This might be good information for developers, but for normal users it is useless. There should be information when to use which one. For example:

Which option takes most time (if there is a significant difference)?
Is one option a superset of the other?
Might one option contain private / secret information which I should not share?

Lets try ‘Run gdb session’. By the way, ‘gdb’ is the GNU project debugger.

Error creating child process

Hrmpf. Seems as if I found a bug while trying to report a bug… This happens for every option.

Ok. Let’s see if I can report the bug. As the automatic tools did not help, I check for a program version via Help > About:

Caja About window

Nice! It is obvious which version I use and how the program is called (Caja 1.8.2). There is even a link to a website where I could eventually report the bug. It links to www.mate-desktop.org.

Report bug

When I search for ‘bug’ on www.mate-desktop.org I get to Reporting Bugs. That seems to be a blog article which explains they are moving to GitHub with a link to github.com/mate-desktop. There are many repositories, so I have to search. Hrmpf. Then I get:

Caja Repositories

5 repositories. Hrmpf. I guess it is simply ‘caja’. When I click on this repository, I have to click on ‘issues’.

Now I am stuck. I don’t know how to find a better description than “it crashed”. There is certainly more information on my system, but I don’t know how to get to it and I don’t see any instructions how to do so. I also don’t want to waste my time searching for information how to get information about the crash.

Google Chrome example

Just a few images…

Step 1
Details 1
Details 2
Details 3
Details 4

What’s wrong and how to fix it

Did you notice how complicated this is for a user? This should be easier. In fact, I think if the bug reporting process is done right it could enhance the trust a user has in software, speed the fixing process up and help develpers to proritize what is important. Some basic steps could be:

The automatic reporting tool should be created for non-developers.
It should automatically store crash information in a place where the user can easily find it (e.g. /home/myaccount/.bugreports/caja/yyyy-mm-dd-hh-mm.xml)
It should be stored in a format which the user can read (e.g. an xml file)
The bug reporting website should be hosted by somebody else.

Now the last one is important in my opinion. It makes sure that users know their bugs are not deleted / removed from the public just because it makes the software look bad. One could add metrics how confident users are, e.g. if you let users tell which software they use. In this case you can add graphs of how many users use the software and how many bugs / issues are reported. (The classification bugs and issues might also not be easy for users!)

Existing Issue Trackers

All of the following bug trackers seem to be developed for developers, not for users:

Open Source Projects:
- Trac: Python
- Bugzilla: Perl
- Mantis Bug Tracker: PHP
- Redmine: Ruby on Rails
Hosted Services:
- GitHub issues (e.g. for numpy)
- Google Code (e.g. for chromium)
- SourceForge (e.g. for dvdstyler)
- Launchpad (e.g. for ubuntu)
- GNU Savannah (e.g. for lordsawar)

See also:

Comparison of source code software hosting facilities

Regular Expressions with Python

info@martin-thoma.de (Martin Thoma) — Tue, 20 Jan 2015 13:35:00 +0100

Python supports regular expressions (RegEx) just as any other general purpose programming language. This mini article shows two examples how to use them.

The package which gives RegEx support is called re.

Replacing Spaces

Replace multiple whitespace characters (spaces, tabs, newlines, …) by a single space (by Nasir):

import re
text = "The   fox jumped   over    the log."
replaced = re.sub("\s\s+", " ", text)

To speed things up you can also compile the pattern. This has also the advantage that you can specify that you want to match newline characters (\n and \r) with the dot by setting the re.DOTALL flag.

import re

pattern = re.compile("\s\s+", re.DOTALL)
text = "The   fox jumped   over    the log."
pattern.sub(" ", text)

Unquoting

import re

def remove_quotes(text):
    """Remove 'test'."""
    def unquote(m):
        return u' %s ' % str(m.group(1))
    pattern = re.compile(" '([^ ]+?.*[^ ]+?)' ")
    return pattern.sub(unquote, text)

text = "The quoted 'text piece' will get unquoted."
remove_quotes(text)

All paragraphs

If you want to get the content within all paragraph tags (<p> ... </p> in arbitrary spacing) you can use:

import re

pattern = re.compile("<p>(.*?)</p>", re.IGNORECASE | re.DOTALL)
text = """
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Phasellus placerat
turpis vitae malesuada tempor. Fusce cursus condimentum leo, ut bibendum justo
varius sit amet. In accumsan interdum nibh at fermentum. Nulla commodo, mi vel
ultricies efficitur, nulla tortor sagittis tortor, quis laoreet augue orci
vitae justo.

sasdf <p>Suspendisse porttitor risus et est consequat condimentum.
In in eleifend ipsum, eu pulvinar nunc. Nullam vel imperdiet eros.
Phasellus vel arcu convallis, semper quam eu, convallis velit.

</p>


<P>Sed tempus magna quis neque varius aliquam. Quisque maximus et augue non
dapibus. Nullam et ante imperdiet, fringilla diam nec, mattis quam.</p>

<P>Ut fermentum lacus id semper imperdiet. Donec et purus consectetur,
fermentum mi nec, viverra dolor. Quisque posuere ultricies leo et efficitur.
Aliquam venenatis quis mi ac tempus.</p>

Maecenas in nisl lacinia, finibus velit eu, aliquet elit. Morbi orci libero,
interdum id vehicula vitae, <p>congue vel</p>p> lectus. Morbi lobortis eros
mollis, cursus velit at, convallis dolor. Nullam volutpat neque at risus
porttitor, faucibus tristique tellus suscipit. In sed purus quis sem tincidunt
lobortis. Vivamus ut blandit erat, sed sollicitudin felis. Etiam placerat,
sapien et vulputate placerat, mi lectus tristique nisi, vitae finibus tellus
nisi a massa. Nullam sit amet scelerisque lectus. Fusce porta justo ac
scelerisque auctor. Integer vestibulum tellus at quam molestie dapibus.
Suspendisse quis quam tortor. Nullam consequat tempus orci eget scelerisque. Ut
auctor lorem enim, id rutrum eros congue varius. In justo lacus, molestie vitae
nibh eu, venenatis auctor ex.
"""

matches = pattern.findall(text)

Matches is the following list (breaked at some points for easier reading):

['Suspendisse porttitor risus et est consequat condimentum.\n
  In in eleifend ipsum, eu pulvinar nunc. Nullam vel imperdiet eros.\n
  Phasellus vel arcu convallis, semper quam eu, convallis velit.\n\n',
 'Sed tempus magna quis neque varius aliquam. Quisque maximus et augue non\n
  dapibus. Nullam et ante imperdiet, fringilla diam nec, mattis quam.',
 'Ut fermentum lacus id semper imperdiet. Donec et purus consectetur,\n
  fermentum mi nec, viverra dolor.
  Quisque posuere ultricies leo et efficitur.\n
  Aliquam venenatis quis mi ac tempus.',
 'congue vel']

Analyzing PyPI Metadata

info@martin-thoma.de (Martin Thoma) — Sun, 18 Jan 2015 20:19:00 +0100

This is part one of a series. See Analyzing PyPI Data - 2 for part two.

PyPI, the Python Package Index, gives a very crappy but simple interface to query metadata about its packages. I scrapped all of the packages metadata. 53,533 packages were scrapped (date: 2015-01-18), because I wanted to see if there is malware on PyPI (related to this question on security.SE).

The database looks like this:

PyPI metadata

Exploring the data

When I scapped the data from PyPI, I made all database fields “varchar 255” as there seems to be no information about the possible values. Then I explored the data

name: The longest package name is 80 characters long (Aaaaaaaaaaa...), the shortest packages have only one character.
author: 911× “UNKOWN”, 741× empty, 195× “None”, 151× “Zope Foundation and Contributors”. There are about 22,000 different authors. There are 595 authors who wrote more than 10 packages.
author_email: 2059× “UNKOWN”, 932× empty, 323× “zope-dev@zope.org”, 216× “None” and 204× “TODO”.
maintainer: 46879× “None”, 5507× empty, 17× Paul Boddie.
requires_python: 53467× “None”, 3× “UNKNOWN”, 2× “>=2.7,!=3.0,!=3.1”, 2× “>=2.5”, 1× “2.6”, 1× “>= 3.3”, 1× “>=2.4”, 1× “>= 2.7” - that’s it. No other values. Seems as if this is pretty much useless.
docs_url: Either begins with http://pythonhosted.org or is empty.

Most active authors

SELECT
    `author`, COUNT(`id`) as `created_packages`
FROM
    `packages`
GROUP BY
    `author`
ORDER BY
    COUNT(`id`) DESC, `author` ASC
LIMIT
    50

gives

	1485
UNKNOWN	1150
None	196
OpenStack	188
Zope Foundation and Contributors	146
MicroPython Developers	139
OpenERP SA	137
Zope Corporation and Contributors	128
RedTurtle Technology	127
Praekelt Foundation	126
Fanstatic Developers	98
Tryton	98
Raptus AG	97
russianidiot	93
hfpython	92
LOGILAB S.A. (Paris, FRANCE)	88
BlueDynamics Alliance	87
Ralph Bean	83
JeanMichel FRANCOIS aka toutpt	69
Bart Thate	68

The total number of authors is 28 183 (06.12.2015):

SELECT COUNT(DISTINCT `author`) AS `total_authors` FROM `packages`

Maximum Length

I guess many values are stored on PyPI as Varchar(255). To check if I might have missed some values, I checked which entry was the longest one for all columns. I did this with

SELECT `name`, LENGTH(`name`) FROM `packages` ORDER BY LENGTH(`name`) DESC

Column	Maximum Length	Entry
name	80	Aaaaaaaaaaaaaaaaaaa-aaaaaaaaa-aaaaaaasa-aaaaaaasa-aaaaasaa-aaaaaaasa-bbbbbbbbbbb
author	248	Michael R. Crusoe, Greg Edvenson, Jordan Fish, Adina Howe, Luiz Irber, Eric McDonald, Joshua Nahum, Kaben Nanlohy, Humberto Ortiz-Zuazaga, Jason Pell, Jared Simpson, Camille Scott, Ramakrishnan Rajaram Srinivasan, Qingpeng Zhang, and C. Titus Brown
author_email	215	ongsp@ucsd.edu, anubhavj@mit.edu, mpkocher@lbnl.gov, geoffroy.hautier@uclouvain.be, wrichard@mit.edu, sdacek@mit.edu, dkgunter@lbl.gov, scholia@lbl.gov, gmatteo@gmail.com, vincentchevrier@gmail.com, armiento@mit.edu
maintainer	63	Panagiotis Liakos, Katia Papakonstantinopoulou, Michael Sioutis
maintainer_email	67	maxistedeams@gmail.com, p_riendeau@live.ca, rheault.etccy@gmail.com
requires_python	17	>=2.7,!=3.0,!=3.1
platform	231	Natural Language :: English License :: OSI Approved :: Apache Software License Environment :: Console Development Status :: 2 - Pre-Alpha Intended Audience :: Developers Programming Language :: Python :: 2.7 Topic :: Database
version	73	[In Progress] Python middle layer for interacting with Redis data easily.
license	466	Copyright © 2014 ЗАО “БАРС Груп” Данная лицензия разрешает лицам, получившим копию данного программного обеспечения и сопутствующей документации (в дальнейшем именуемыми «Программное Обеспечение»), безвозмездно использовать Программное Обеспечение без ог
keywords	655	ArcLink,array,array analysis,ASC,beachball,beamforming,cross correlation,database,dataless,Dataless SEED,datamark,earthquakes,Earthworm,EIDA,envelope,events,FDSN,features,filter,focal mechanism,GSE1,GSE2,hob,iapsei-tau,imaging,instrument correction,instrument simulation,IRIS,magnitude,MiniSEED,misfit,mopad,MSEED,NERA,NERIES,observatory,ORFEUS,picker,processing,PQLX,Q,real time,realtime,RESP,response file,RT,SAC,SEED,SeedLink,SEG-2,SEG Y,SEISAN,SeisHub,Seismic Handler,seismology,seismogram,seismograms,signal,slink,spectrogram,StationXML,taper,taup,travel time,trigger,VERCE,WAV,waveform,WaveServer,WaveServerV,WebDC,web service,Winston,XML-SEED,XSEED
description	65535	[id don't want to put that here - there were 22 with this length]
summary	278	Splits one vcard file (*.vcf) to many vcard files with one vcard per file. Useful for import contacts to phones, thats do not support multiple vcard in one file. Supprt unicode cyrillic characters. Консольная программа для р
stable_version	4	None
home_page	134	http://127.0.0.1:8888/USK@9X7bw5HD2ufYvJuL3qAVsYZb3KbI9~FyRu68zsw5HVg,lhHkYYluqHi7BcW1UHoVAMcRX7E5FaZjWCOruTspwQQ,AQACAAE/pyfcp-api/0/
release_url	130	http://pypi.python.org/pypi/softwarefabrica.django.appserver/1.0dev-BZR-r10-panta-elasticworld.org-20091023132843-vitk6k7e5qlvhej5
bugtrack_url	104	https://bugzilla.redhat.com/buglist.cgi?submit&component=python-nss&product=Fedora&classification=Fedora
download_url	183	http://pypi.python.org/packages/source/s/softwarefabrica.django.appserver/softwarefabrica.django.appserver-1.0dev-BZR-r10-panta-elasticworld.org-20091023132843-vitk6k7e5qlvhej5.tar.gz
package_url	108	http://pypi.python.org/pypi/Aaaaaaaaaaaaaaaaaaa-aaaaaaaaa-aaaaaaasa-aaaaaaasa-aaaaasaa-aaaaaaasa-bbbbbbbbbbb
_pypi_hidden	-	True or False
_pypi_ordering	-	Integers from 0 to 464
cheesecake_code_kwalitee_id	-	Integers from 183 to 6513 and None values
cheesecake_documentation_id	-	Integers from 182 to 6512 and None values
cheesecake_installability_id	-	Integers from 181 to 6511 and None values

We can see multiple problems here:

URLs: localhost / 127.0.0.1 is almost certainly not desired
cheesecake_installability_id, cheesecake_documentation_id, cheesecake_code_kwalitee_id should have NULL and None should be casted to NULL. The data type is likely to be numeric.
_pypi_ordering should be numeric
_pypi_hidden should be boolean

So I changed the types in the database. Let’s continue.

What people don’t use

All metadata is provided by the authors of the packages. Some fields, like the package name or the description, are used very often, but some fields are only rarely used:

Maintainer

maintainer and maintainer_email is interesting for people who want to send bug reports. If this is empty, I would guess the package is dead.

Platform

SELECT
    `platform`, COUNT(`id`)
FROM
    `packages`
GROUP BY
    `platform`
ORDER BY
    COUNT(`id`) DESC

gives 759 different results. The TOP-10 were

Platform	Count
UNKNOWN	43539
any	3515
	1986
Any	635
OS Independent	542
None	265
Linux	225
linux	167
Posix; MacOS X; Windows	157
POSIX	156

You can see that there are alternative ways to express the same thing. Also, the “;” should not be here as the manual states it should be a list.

The manual is also too short for this entry. It only says “a list of platforms” which seems to be pretty much useless.

docstore.mik.ua gives a little bit more information

A list of platforms on which this distribution is known to work. You should provide this information if you have reasons to believe this distribution may not work everywhere. This information should be reasonably concise, so this field may refer for details to a file in the distribution or to a URL.

Bugtracker

This one is very important. Users should have an easy possibility to report bugs. So please help them by added your bug tracker URL wherever it makes sense. Here is how you add it on PyPI:

Go to your packages PyPI page

Add your bugtracker / issue tracker url

Check if you really added it

License

Another important one. Add a license to your software!

SELECT
    `license`, COUNT(`id`)
FROM
    packages
GROUP BY
    `license`
ORDER BY
    COUNT(`id`) DESC

gives

License	Count
UNKNOWN	11444
MIT	8098
BSD	7224
GPL	4202
MIT License	1410
(empty)	1398
LICENSE.txt	1201
GPLv3	1083
LGPL	833
ZPL 2.1	829
BSD License	811
Apache License 2.0	482
Apache License, Version 2.0	443
Apache 2.0	437
GPLv2	370
None	363
ZPL	288
GPL-3	272
GPLv3+	258
LICENSE	249

and about 6500 other licenses. Most might be variants in writing, e.g.

Apache License 2.0
Apache License, Version 2.0
Apache 2.0

are all the same license. Some packages have no licence (Unknown and empty). Many are invalid values, like

LICENSE.txt
None
LICENSE
Python

These indicate that people have no idea what to input there. tldrlegal.com might help in that case. I think the Python community should try to eliminate variants in writing license names as it makes finding, filtering and analyzing packages more difficult.

Classifiers

Python makes use of so called trove classifiers. They are defined in PEP 301 and listed here.

The following table gives the TOP-10 most commonly used classifiers:

Classifier	Percentage
Programming Language :: Python	0.4615
Intended Audience :: Developers	0.4412
Operating System :: OS Independent	0.3229
Programming Language :: Python :: 2.7	0.2183
Topic :: Software Development :: Libraries :: Pyth…	0.2038
Development Status :: 4 - Beta	0.2038
License :: OSI Approved :: BSD License	0.1632
License :: OSI Approved :: MIT License	0.1529
Environment :: Web Environment	0.1487
Programming Language :: Python :: 2.6	0.1306

When you analyze the licenses with the trove classifiers you get a different image:

SELECT
    `classifier`, COUNT(`id`) / 53533
FROM
    `package_classifiers`
WHERE
    `classifier` LIKE "License%"
GROUP BY
    `classifier`
ORDER BY
    COUNT(`id`) DESC

License	Percentage
License :: OSI Approved :: BSD License	0.1632
License :: OSI Approved :: MIT License	0.1529
License :: OSI Approved :: GNU General Public License (GPL)	0.0625
License :: OSI Approved :: Apache Software License	0.0460
License :: OSI Approved :: GNU Library or Lesser General Public License (LGPL)	0.0188

There are 94 trove classifiers with five or less packages which use this classifier. I guess many of them are not in the official list of classifiers.

Package type

SELECT
    `packagetype`, COUNT(`id`)
FROM
    `urls`
GROUP BY
    packagetype
ORDER BY
    COUNT(`id`) DESC

gives

Package type	Count
sdist	47649
bdist_egg	4933
bdist_wheel	3098
bdist_wininst	1512
bdist_dumb	577
bdist_msi	45
bdist_rpm	35
bdist_dmg	4

Can anybody explain what this means?

Downloads

SELECT
    `name`, `url`, `downloads`
FROM
    `urls`
JOIN
    `packages` ON `urls`.`package_id` = `packages`.`id`
ORDER BY
    `urls`.`downloads`  DESC
LIMIT 10

gives

Package type	Count
wincertstore	10026403
ssl	8987455
pyasn1	8655361
Paste	8401111
PyYAML	7180547
distribute	6276421
MarkupSafe	6215382
ecdsa	6030395
meld3	5715687
pika	5567400

and

SELECT
    `name`, SUM(`downloads`)
FROM
    `releases`
JOIN
    `packages` ON `releases`.`package_id` = `packages`.`id`
GROUP BY
    `name`
ORDER BY
    SUM(`downloads`)  DESC
LIMIT 10

Package type	Count
setuptools	45485205
requests	35446321
virtualenv	35039299
distribute	34779943
boto	29678066
six	28253705
certifi	27381407
pip	26266325
wincertstore	24831145
lxml	20901298

Size

What is the biggest Python package?

SELECT
    `name`, `release_number`, `size`
FROM
    `releases`
JOIN
    `packages` ON `releases`.`package_id` = `packages`.`id`
ORDER BY
    `releases`.`size` DESC
LIMIT 30

Package	Version	Size
de422	2009.1	545298406
de406	1997.1	178260546
scipy	0.13.3	62517637
appdynamics-bindeps-linux-x86	5913-master	57861536
appdynamics-bindeps-linux-x64	5913-master	56366211
python-qt5	0.1.5	56259023
python-qt5	0.1.8	56237972
pycalculix	0.92	56039839
wltp	0.0.9-alpha.3	55414544
cefpython3	31.2	55163815

Code

See github.com/MartinThoma/algorithms.

Further Ideas

Build a dependency graph: Some of the code was already written. However, one has to download about 25 GB of data, extract it and run over those files. This is quite a bit of work.
Analyze package quality
- Missing requirements
- Missing metadata / description
- Missing documentation
- PEP8
- Code duplication
Malicious package search:
- Check which package names are prefixes of other package names.
- Find packages which upload data (dropbox?)
- Find pacakges which remove data from your file system

LaTeX and tables

info@martin-thoma.de (Martin Thoma) — Tue, 30 Dec 2014 21:57:00 +0100

This is a quick article I had for quite a while as a draft. It might not be finished or have other problems, but I still want to share it.

LaTeX is an awesome typesetting language: It is powerful enough to let you do anything you want, it has a great community and standard solutions for every common problem as it is quite old. But tables… well, let’s say there is much room for improvement.

How to make tables

The simple way

The most basic table I could think of is generated like this:

\documentclass[a4paper]{scrartcl}
\usepackage{amssymb, amsmath}  % needed for math
\begin{document}

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis malesuada 
efficitur volutpat. Proin eget dui in lacus fermentum pharetra. Sed ut 
facilisis leo, sed consectetur urna. Phasellus eget tempus eros, vitae 
sagittis mi. Nullam pharetra, dolor a efficitur blandit, nisl velit interdum 
turpis, vel posuere lorem justo nec elit. Donec mattis massa auctor mattis 
varius. Nunc sed volutpat sapien, at finibus lectus. Phasellus auctor vehicula 
auctor. Nam vitae purus ut orci accumsan auctor. Fusce volutpat, tortor 
consectetur convallis aliquam, enim velit gravida sapien, id convallis nisi 
nunc ac leo.

    \begin{table}[ht]
        \centering
        \begin{tabular}{l|rrr}
        Country / Property & Population       & Area     & HDI \\\hline
        France             & $66 \cdot 10^6$  & $668763\text{km}^2$  & 0.89 \\
        Germany            & $81 \cdot 10^6$  & $357167\text{km}^2$  & 0.92 \\
        United States      & $317 \cdot 10^6$ & $9629091\text{km}^2$ & 0.94 \\
        \end{tabular}
        \caption{Information about countries}
        \label{table:countries}
    \end{table}

Aenean sit amet felis eu urna accumsan consectetur. Sed hendrerit ultrices 
turpis nec fringilla. Pellentesque sagittis neque placerat, pharetra ante 
quis, maximus eros. Nam ultrices lacinia magna, eget interdum tortor ornare 
ut. Praesent semper tristique consectetur. Sed auctor, orci accumsan imperdiet 
vestibulum, nunc augue sagittis purus, sed dapibus neque arcu sit amet enim. 
Aliquam fermentum dui eu efficitur condimentum. Maecenas viverra metus ut 
bibendum pellentesque. Integer non interdum massa. Vestibulum non enim 
vulputate, sodales nunc ac, ultricies tortor. Suspendisse non vehicula ipsum, 
quis consequat elit. Quisque rutrum tincidunt lorem id semper. Phasellus 
bibendum nulla sit amet purus tempus, vitae tincidunt ligula ornare. Nunc quis 
felis non ex consequat elementum nec sit amet magna. In lacinia nulla nec neque 
venenatis, vel tristique risus blandit.
\end{document}

It will look like this:

Standard LaTeX table

You can see two environments: table and tabular. Let’s focus on tabular first. It is always followed by a list of characters l, c, r which define if the column is left-aligned, centered or right-aligned. You can add | to tell LaTeX if there should be a line to distinguish columns. After that you get to the first cells. & starts a new cell and \\ starts a new line. If you want a drawn line / rule, you can use \hline.

Let’s check out table: table has options after it which are commonly ht where h means “here” and t means “top”. So LaTeX tries to place the table where it is in the text and if that doesn’t work out it places the table on the top of the page. It can get a caption \caption{Description of the contents of the table} and a \label{table:your-label-for-internal-usage}.

The standard way

I always use \usepackage{booktabs} to get \toprule, \midrule and \bottomrule:

\documentclass[a4paper]{scrartcl}
\usepackage{amssymb, amsmath}  % needed for math

\usepackage{booktabs}  % for \toprule, \midrule and \bottomrule
\begin{document}

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis malesuada 
efficitur volutpat. Proin eget dui in lacus fermentum pharetra. Sed ut 
facilisis leo, sed consectetur urna. Phasellus eget tempus eros, vitae 
sagittis mi. Nullam pharetra, dolor a efficitur blandit, nisl velit interdum 
turpis, vel posuere lorem justo nec elit. Donec mattis massa auctor mattis 
varius. Nunc sed volutpat sapien, at finibus lectus. Phasellus auctor vehicula 
auctor. Nam vitae purus ut orci accumsan auctor. Fusce volutpat, tortor 
consectetur convallis aliquam, enim velit gravida sapien, id convallis nisi 
nunc ac leo.

    \begin{table}[ht]
        \centering
        \begin{tabular}{l|rrr}
        \toprule
        Country / Property & Population       & Area     & HDI \\\midrule
        France             & $66 \cdot 10^6$  & $668763\text{km}^2$  & 0.89 \\
        Germany            & $81 \cdot 10^6$  & $357167\text{km}^2$  & 0.92 \\
        United States      & $317 \cdot 10^6$ & $9629091\text{km}^2$ & 0.94 \\
        \bottomrule
        \end{tabular}
        \caption{Information about countries}
        \label{table:countries}
    \end{table}

Aenean sit amet felis eu urna accumsan consectetur. Sed hendrerit ultrices 
turpis nec fringilla. Pellentesque sagittis neque placerat, pharetra ante 
quis, maximus eros. Nam ultrices lacinia magna, eget interdum tortor ornare 
ut. Praesent semper tristique consectetur. Sed auctor, orci accumsan imperdiet 
vestibulum, nunc augue sagittis purus, sed dapibus neque arcu sit amet enim. 
Aliquam fermentum dui eu efficitur condimentum. Maecenas viverra metus ut 
bibendum pellentesque. Integer non interdum massa. Vestibulum non enim 
vulputate, sodales nunc ac, ultricies tortor. Suspendisse non vehicula ipsum, 
quis consequat elit. Quisque rutrum tincidunt lorem id semper. Phasellus 
bibendum nulla sit amet purus tempus, vitae tincidunt ligula ornare. Nunc quis 
felis non ex consequat elementum nec sit amet magna. In lacinia nulla nec neque 
venenatis, vel tristique risus blandit.
\end{document}

which looks like this:

LaTeX table with booktabs

Combining cells

It is much nicer when you combine cells. To do so, you can use \multicolumn:

\documentclass[a4paper]{scrartcl}
\usepackage{amssymb, amsmath}  % needed for math

\usepackage{booktabs}  % for \toprule, \midrule and \bottomrule
\begin{document}
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis malesuada 
efficitur volutpat. Proin eget dui in lacus fermentum pharetra. Sed ut 
facilisis leo, sed consectetur urna. Phasellus eget tempus eros, vitae 
sagittis mi. Nullam pharetra, dolor a efficitur blandit, nisl velit interdum 
turpis, vel posuere lorem justo nec elit. Donec mattis massa auctor mattis 
varius. Nunc sed volutpat sapien, at finibus lectus. Phasellus auctor vehicula 
auctor. Nam vitae purus ut orci accumsan auctor. Fusce volutpat, tortor 
consectetur convallis aliquam, enim velit gravida sapien, id convallis nisi 
nunc ac leo.

\begin{table}[ht]
    \centering
    \begin{tabular}{llll}
    \toprule
    Country & \multicolumn{3}{c}{Property}\\
              \cmidrule{2-4}
            & {Population} & {Area} & {HDI} \\
            & {(mio.)} & {(km\textsuperscript{2})}\\
    \midrule
    France             & \multirow{2}{c}{66}  & 668763  & 0.89 \\
    Germany            & 81  & 357167  & 0.92 \\
    United States      & 317 & 9629091 & 0.94 \\
    \bottomrule
    \end{tabular}
    \caption{Information about countries}
    \label{table:countries}
\end{table}
Aenean sit amet felis eu urna accumsan consectetur. Sed hendrerit ultrices 
turpis nec fringilla. Pellentesque sagittis neque placerat, pharetra ante 
quis, maximus eros. Nam ultrices lacinia magna, eget interdum tortor ornare 
ut. Praesent semper tristique consectetur. Sed auctor, orci accumsan imperdiet 
vestibulum, nunc augue sagittis purus, sed dapibus neque arcu sit amet enim. 
Aliquam fermentum dui eu efficitur condimentum. Maecenas viverra metus ut 
bibendum pellentesque. Integer non interdum massa. Vestibulum non enim 
vulputate, sodales nunc ac, ultricies tortor. Suspendisse non vehicula ipsum, 
quis consequat elit. Quisque rutrum tincidunt lorem id semper. Phasellus 
bibendum nulla sit amet purus tempus, vitae tincidunt ligula ornare. Nunc quis 
felis non ex consequat elementum nec sit amet magna. In lacinia nulla nec neque 
venenatis, vel tristique risus blandit.
\end{document}

which looks like this:

LaTeX table with multicols

Note the \cmidrule{2-4} which draws a rule from cell 2 to 4 (LaTeX starts to count at 1).

If you want to combine multiple rows you need to use \usepackage{multirow}.

Page breaking tables

If you want your tables to be able to break over pages, you have to use \usepackage{longtable} which has an example.

Another option is supertabular.

When you use them, you should add \endhead so that the head can get repeated.

Tools

truben.no/table is an awesome page to create tables for different formats.

What could be better

Page-breaking: I understand that you don’t want to have page breaking of tables most of the time. But rather than never allowing page-breaking, I would expect an optimization with the following thoughts in mind:

It is best to have the table where it was specified.
The content should have at least 2 times as many rows as the header. So when there is only one header row, there should be at least 2 content rows.
The farer a table is away from its “original” position, the worse it is.

Headers: A semantic way to define the header and a tail would be very good. \endhead is ok.

Semantics, Styles and Classes: LaTeX hides some of the formatting for semantic codes. For example, you write \section{MySection} instead of {\fontsize{12}{15}\textbf{MySection}} or something similar. That should be more often the case. In fact, I think it would be very nice if LaTeX had some built-in support for pure stylesheets (like CSS for HTML). It would be very nice if I only had to define that I want to make a standard table and it inserts \toprule, \midrule and \bottomrule automatically. But to get an automatic \midrule we need a command to tell LaTeX where the header ends.

Everything of the above should be standard. There should not be the need to use new packages for that. Also, \multirow, \toprule, \midrule, \bottomrule should not be “hidden” in a package but be there by default.

Table seperator: It is unfortunate that the table cell seperator(s) \\ and & have problems with matrices. I think one environment should get closed before the other one can continue parsing, so I don’t quite understand where the problem is. But there certainly is a problem (see this question).

Tools: I don’t know any tools that can export LaTeX tables and merge cells. Such tools should be able to import and export LaTeX tables

Hörspiele

info@martin-thoma.de (Martin Thoma) — Tue, 30 Dec 2014 21:51:00 +0100

This is a quic article I had for quite a while as a draft. It might not be finished or have other problems, but I still want to share it.

Ich habe früher (damals, als ich noch keinen PC hatte ☺ ) sehr gerne Hörspiele gehört. Nun, mit meinem neuen Smartphone, dachte ich könnte ich das ja mal wieder machen.

Auf der Suche nach (kostenlosen) Hörspielen habe ich folgendes gefunden:

Neuvertonung.de: Eine kleine Gruppe hat „Die drei ???“ neu vertont. Eine Liste der Folgen gibts hier.
YouTube

Trickreiche Matheaufgaben

info@martin-thoma.de (Martin Thoma) — Tue, 30 Dec 2014 21:30:00 +0100

This is a quick article I had for quite a while as a draft. It might not be finished or have other problems, but I still want to share it.

Ich finde immer wieder ein paar einfache, aber irgendwie trickreiche Matheaufgaben. Was genau ich damit meine, seht ihr am besten am folgendem Beispiel.

Wechselgeld

Aufgabe

Drei Kinder haben je 10 Euro und kaufen sich dafür gemeinsam einen Ball für 30 Euro. Nach dem Verkauf stellt der Ladeninhaber fest, dass der Ball nur 25 Euro kostet und schickt seinen Lehrling mit den überzähligen 5 Euro den Kindern nach. Der Lehrling gibt jedem Kind 1 Euro und behält für seine Bemühungen die restlichen 2 Euro. Damit hat jedes Kind nur 9 Euro für den Ball bezahlt, insgesamt zahlten die Kinder 27 Euro. Mit den 2 Euro des Lehrlings ergibt das aber erst 29 Euro.

Quelle: dsm-faq.wikidot.com/denksport

Auflösung

Das Problem dieser Aufgabe ist, dass man sich nicht klar macht, wo man hin will. Man will scheinbar herausfinden, wo das Geld geblieben ist. Das vermischt man aber mit dem Geld, das man hatte:

Situation vor dem Kauf:

Kinder: je 10 Euro
Lehrling: 0 Euro
Ladeninhaber: 0 Euro

3 Kinder: je 1 Euro
Lehrling: 2 Euro
Ladeninhaber: 25 Euro

Mücke und Elefant

Aufgabenstellung

Sei `$x$` das Gewicht des Elefanten und `$y$` das Gewicht der Mücke. Sei `$d$` der Unterschied.
`$\begin{align} x &= y + d | \cdot (x-y)\\ x^2 - xy &= xy + xd - y^2 - yd | -xd \\ x^2 - xy - xd &= xy - y^2 - yd\\ x(x-y-d) &= y \cdot (x-y-d) |:(x-y-d)\\ x &=y \end{align}$` Das Gewicht der Mücke ist also gleich dem Gewicht des Elefanten!

dsm-faq.wikidot.com/denksport

Auflösung

Jeder Mensch hat sein Idealgewicht

Aufgabenstellung

Auflösung

Vertex coloring

info@martin-thoma.de (Martin Thoma) — Tue, 30 Dec 2014 21:29:00 +0100

This is a quick article I had for quite a while as a draft. It might not be finished or have other problems, but I still want to share it.

The problem

The Vertex Coloring problem can be described like this:

Vertex Coloring Let `$G = (V, E)$` be an undirected graph. Find a function `$f: V \rightarrow \{1, \dots, n\}, n \in \mathbb{N}$` such that: `$\forall e=\{v_1, v_2\} \in E: f(v_1) \neq f(v_2)$`. Minimize `$n$`.

If $n$ is minimal for $G$ , it is called the chromatic number $\chi(G)$ .

What does that mean?

You have a graph. Then you take pencils and color the vertices such all vertices that are connected are not of the same color.

Example

All of the following graphs show valid thee colorings:

[caption id=”attachment_69831” align=”aligncenter” width=”500”] Valid thee colorings of one graph
Source: Wikipedia[/caption]

Interesting facts

`$1 \leq \chi(G) \leq |V|$`: You need at least one color and you could color all vertices with a different color
`$\chi(K_n) = n$`: All complete graphs with `$n$` vertices need exactly `$n$` colors. One color for each vertex.
Every planar graph can be colored with 4 colors (see four color theorem).
Determining if a graph can be colored with 2 colors is equivalent to determining whether or not the graph is bipartite. This can be checked in polynomial time. You simply start with one vertex, give it color 1 and all adjacent vertices color 2. Then all adjacent vertices of color 2 have to have color 1, ...
Vertex Coloring is in `$\mathcal{NPC}$`.
All triangle-free planar graphs can be 3-colored. You can get this coloring in linear time (source).

Algorithms

I think learning from errors is important. This is the reason why I share the following algorithm that do not work.

First WRONG try: Fix adjacent vertices

[caption id=”attachment_69891” align=”aligncenter” width=”512”] A vertex coloring algorithm that does not work[/caption]

The time complexity of this algorithm is in $\mathcal{O}(|V|^2)$ . This should make you suspicious, as Vertex Coloring is in $\mathcal{NPC}$ . So if it was correct, it would solve the P vs. NP problem which is worth a million dollars.

But an example why it doesn’t work is better. Just try it for the following graph:

[caption id=”attachment_69921” align=”aligncenter” width=”512”] Example that does not work with the provided algorithm[/caption]

Second WRONG try: Fix current vertex

[caption id=”attachment_69971” align=”aligncenter” width=”512”] Another vertex coloring algorithm[/caption]

This algorithm gives a valid coloring, but the coloring is not minimal.

Example:

When you apply the algorithm the the graph below, you will get a coloring with four colors. But obviously, it is possible to color it with three colors. [caption id=”attachment_69951” align=”aligncenter” width=”512”] Graph that can be vertex-colored with 3 colors[/caption]

Brute force

It’s always a good idea to think about brute force algorithms. On the one hand, they are simple. So you can intuitively understand why they are correct and see their time / space complexity. On the other hand, you can use them for sanity checks of better algorithms for small problem instances.

Here is a brute force algorithm for the vertex coloring problem [caption id=”attachment_70001” align=”aligncenter” width=”512”] Brute force a minimal vertex coloring[/caption]

You need $\sum_{i=2}^n i^n$ steps at maximum. Wow. This is MUCH. Even when you only want to check if $i$ colors are enough, you need $i^n$ .

You could use the second algorithm to get a better upper bound and try the next smaller ones. As soon as you don’t get valid colorings, you know that the number of colors you’ve used in the last valid coloring was the minimum number.

Zero-knowledge protocol

Vertex coloring is relevant for so called “zero-knowledge protocols”. This is a method by which one party (the prover) can prove to another party (the verifier) that a given statement is true, without conveying any additional information apart from the fact that the statement is indeed true.

A zero-knowledge proof must satisfy three properties:

Completeness: if the statement is true, the honest verifier (that is, one following the protocol properly) will be convinced of this fact by an honest prover.

Soundness: if the statement is false, no cheating prover can convince the honest verifier that it is true, except with some small probability.

Zero-knowledge: if the statement is true, no cheating verifier learns anything other than this fact. This is formalized by showing that every cheating verifier has some ''simulator'' that, given only the statement to be proven (and no access to the prover), can produce a transcript that "looks like" an interaction between the honest prover and the cheating verifier.

Source: Wikipedia

Another great example is the following:

Imagine your friend is color-blind. You have two billiard balls; one is red, one is green, but they are otherwise identical. To your friend they seem completely identical, and he is skeptical that they are actually distinguishable. You want to prove to him (I say "him" as most color-blind people are male) that they are in fact differently-colored. On the other hand, you do not want him to learn which is red and which is green. Here is the proof system. You give the two balls to your friend so that he is holding one in each hand. You can see the balls at this point, but you don't tell him which is which. Your friend then puts both hands behind his back. Next, he either switches the balls between his hands, or leaves them be, with probability 1/2 each. Finally, he brings them out from behind his back. You now have to "guess" whether or not he switched the balls. By looking at their colors, you can of course say with certainty whether or not he switched them. On the other hand, if they were the same color and hence indistinguishable, there is no way you could guess correctly with probability higher than `$\frac{1}{2}$`. If you and your friend repeat this "proof" `$t$` times (for large `$t$`), your friend should become convinced that the balls are indeed differently colored; otherwise, the probability that you would have succeeded at identifying all the switch/non-switches is at most `$2^{-t}$`. Furthermore, the proof is "zero-knowledge" because your friend never learns which ball is green and which is red; indeed, he gains no knowledge about how to distinguish the balls.

Source: mathoverflow.net

Lets make this more concrete. Say you want to authenticate somebody. This person is identified as the “only” person who knows a three-coloring of a big graph. This makes him/her special. If you simply asked him “what’s the three coloring for your graph?” he would no longer be the only person who knows the three coloring.

So you want to get sure that he knows a three coloring without getting it.

Warum ist Q abzählbar?

info@martin-thoma.de (Martin Thoma) — Tue, 30 Dec 2014 21:27:00 +0100

This is a quick article I had for quite a while as a draft. It might not be finished or have other problems, but I still want to share it.

Eine Menge B heißt abzählbar `$: \Leftrightarrow \exists (a_n) \in B: B = \{a_1, a_2, a_3, ...\}$` `$\Leftrightarrow \exists f : \mathbb{N} \rightarrow B $` mit `$f$` surjektiv.

Die natürlichen Zahlen sind abzählbar. Beh.: $\mathbb{N}$ ist abzählbar. Bew.: direkt Sei $f: \mathbb{N} \rightarrow \mathbb{N}$ definiert durch $f(n) := n$ . $f$ ist also die identität und damit bijektiv und insbesondere surjektiv $\blacksquare$

Die ganzen Zahlen sind abzählbar. Beh.: $\mathbb{Z}$ ist abzählbar. Bew.: direkt Sei $f: \mathbb{N} \rightarrow \mathbb{Z}$ definiert durch $f(n) := \begin{cases} \frac{n-1}{2} & \text{, falls n ungerade} \\ - \frac{n}{2} & \text{, falls n gerade} \end{cases}$ . Also: $\forall z \in \mathbb{Z}: \exists x \in \mathbb{N} : f(x) = z$ da: $\forall x \in \mathbb{Z}: n = \begin{cases} - 2 \cdot x & \text{, falls x negativ} \\ 2 \cdot x + 1 & \text{, falls x positiv} \end{cases}$

Es gibt also für jede ganze Zahl z eine natürliche Zahl n, die ich in $f$ stecken kann um z zu erhalten $\blacksquare$

Beh.: $\mathbb{N} \times \mathbb{N}$ ist abzählbar. Bew.: direkt Sei $f: \mathbb{N} \times \mathbb{N} \rightarrow \mathbb{N}$ rekursiv definiert durch $f(1,1) := 1$ und $f (m, n) := \begin{cases} f(m + 1, n - 1) + 1 & \text{, falls } n \neq 1 \\ f(1, m - 1) & \text{sonst} \end{cases}$ Diese Abbildung sieht wie folgt aus: [caption id=”attachment_20501” align=”aligncenter” width=”322” caption=”Abbildung, die N x N auf N abbildet”][/caption]

Ich finde es ist intuitiv klar, dass diese Funktion bijektiv ist. Hat jemand dafür einen sauberen Beweis?

Also gibt es eine Umkehrfunktion (die auch bijektiv ist). Also ist $\mathbb{N} \times \mathbb{N}$ abzählbar $\blacksquare$

Beh.: $\mathbb{Q}^+$ ist abzählbar. Bew.: über $N \times N$ Jede Zahl $x \in \mathbb{Q}^+$ kann mit zwei natürlichen Zahlen dargestellt werden: $x = \frac{p}{q}$ . Also gibt es eine Funktion $f: \mathbb{N} \times \mathbb{N} \rightarrow \mathbb{Q}$ mit $f(m, n) := \frac{m}{n}$ . Diese Abbildung ist offensichtlich surjektiv. $\blacksquare$

Material

Die LaTeX-Dateien für die Bilder sind in diesem Archiv zu finden.

Why I prefer Linux over Windows

info@martin-thoma.de (Martin Thoma) — Tue, 30 Dec 2014 21:25:00 +0100

This is a quick article I had for quite a while as a draft.It might not be finished or have other problems, but I still want to share it.

Some friends wondered why I prefer Linux over Windows. As I am currently using only Linux, I can make some examples.

Here is an open list, why I prefer Linux over Windows:

No marketing strategies

Windows 7 comes in may flavours: Windows 7 Home, Windows 7 Home Premium, Windows 7 Ultimate, Windows 7 Enterprise, Windows 7 Professional, … (see Microsoft product flavor hell)

I have to admit that the choice of Linux can be difficult, too. You can choose from different distributions like Ubuntu, Suse, Fedora, … (see distrowatch.com) and some distributions offer different desktop environments like GNOME and KDE. The important difference is that Linux flavors depend on your needs, but Windows flavors depend on your money.

User-friendly system

As a Ubuntu 10.04 LTS user, I think that Ubuntu is much more user friendly than Windows 7. You have much more control about your system than you have on Windows (see Why are Microsoft products so User unfriendly?)

Here are some everyday examples:

Chaning the sound volume on a Notebook: You will see an indicator in Ubuntu like this. On Windows, you have to guess or wait until your movie starts
Taking a screenshot: In Ubuntu, you only have to press "Print Screen". On Windows, you have to know Snipping tool or install some additional software. Additionally, it seems not to be possible to get a the key "Print Screen" as a shortcut for taking screenshots (source).
Different workspaces, pinning a window to "always in foreground" is definitely missing in Windows.
PDF-Printers: oh my god. This is really sad.

Better community

When I have questions for my system, I can ask them on askubuntu.com, unix.stackexchange.com or on ubuntuusers.de. I usually get friendly answers that help me to fix my problem within a few minutes.

Where do I find answers to Windows questions? I have tried superuser.com, but is there anything else?

Repositories

When I want to install something on my Ubuntu machine, I simply type:

sudo apt-get install something

When I want to install something on Windows, I have to Google for it. When I find a tool which seems to fit, I have to find out if it is for free or if it’s only a trial version. Then I need to find a way to download it and make sure that it’s not malware.

When I want to update all software I have on my linux machine, I type:

sudo apt-get update
sudo apt-get upgrade

After this command, my system and every single piece of software I have installed is updated. The updates are automatically installed in the background. A restart might be necessary, but until the restart is done the old software is used.

On Windows, I have to:

Click on the start button
Type "update" in the search bar
Click on "Search for updates"
Install all updates

A restart might be required. But I can’t simply make the restart when I want to. No, on Windows you will get reminded. You can choose the delay (max. 4 hours) of the reminder, but you can’t disable it. And this is only an update for the operating system. You have to look for updates of your software by yourself. For every single piece of software! This is not so easy. How do you find a reliable source of Updates e.g. for Unreal Tournament 2004?

Terminal

You can do everything with terminal. When the system is slowing down, I press Ctrl + Alt + F4, log into the shell, call top and kill the process which slows my system down.

And I really like ZSH and Oh-My-ZSH.

Linux is gratis

You don’t have to pay for it. In comparison, Windows 7 costs now (31.12.2014) about 50 Euro on Amazon. Although it is already outdated.

Simple stuff

There are some simple, little things which I like when I use Linux. I can’t name them all, but some that come to my mind are:

LiveCD

Isn’t it great to have the possibility to use the OS from a CD / DVD only? This gives you the possibility to check if your system runs (or to diagnose what’s going wrong) without chaning anything.

Installation setup

The installation setup is great. It detects at the beginning if everything is ok (disk space, internet connection, battery) and tells you in simple words what is wrong. To chose your time zone you are shown a very simple graphic and by now the default was always correct for me. It continues with keyboard detection. Although the default was always wrong for me by now, it has an awesome auto-detection tool. You simply have to type some letters and it returns your layout. Great!

Reasons to stay with Windows

Although I don’t like Windows 7 for many reasons, I can see some reasons to stay with Windows:

Linux doesn’t support your hardware (see Check Computer / Hardware for Linux-compatibility)
Linux doesn’t support your software AND no free alternative exists (e.g. Photoshop for professionals)

Pseudo reasons for Linux

A pseudo-reason is an argument which might be true, but is not important at all for the person who wrote it.

Security

I often hear that people like Linux because of higher security. I don’t think that this is a real reason, as I have never heared of any end user having switched because of security reasons. I also don’t think that there is a significant difference of the bare systems in security.

Freedom to change code

Some people argue, that you can change the code of Linux / OpenSource programs according to your needs. This is only an argument, if you have done it at least once.

Pseudo reasons against Linux

Linux is only for geeks

This is obviously not true. I know at least some non-geeks who are able to use it.

Linux supports NO games!

Not true either. Steam gives A LOT of high quality games to Linux and you also have the possibility to use wine.

However, if you want a specific game that might be a different story.

Don't lose your stuff

info@martin-thoma.de (Martin Thoma) — Mon, 29 Dec 2014 21:07:00 +0100

This is a quick article I had for quite a while as a draft. It might not be finished or have other problems, but I still want to share it.

It is very annoying when you buy something expensive and you lose it or it gets stolen. But there are some technical solutions to prevent that.

Crowd GPS Tags

Crowd GPS devices use Bluetooth and connect with Smartphones. Whenever another person has the App installed, he/she sends the location of all devices to a server. This means as long as anybody is near the GPS tag, you can hope to find your stuff again.

Tile

Tile is such a crowd GPS device.

There is no android app yet.

TinTag

TinTag is a 28mm × 39.2mm device. Its battery lasts for 4 months. It has an LED and a buzzer as well as bluetooth. The bluetooth connection lasts for 100m.

It has a indigogo campagain. 10 Tintag devices and 1 base charger cost 105 US-Dollar.

Tintag app works on Android phones (version 4.3 and 4.4) and iPhone. It doesn’t work yet on Windows Phones or Blackberry.

TrackR bravo

TrackR bravo is just another tagging device.

TrackR is compatible with Android 4.4 (see App) and iPhone 4s & later, iPad 3rd Generation & later. It works with Bluetooth 4.0. It uses a CR1616 coin cell battery.

It is 31mm in diameter and 3.5mm height.

It can detect bluetooth devices up to 30m.

One TrackR costs 29 US-Dollar, 10 cost 99 US-Dollar.

StickNFind

StickNFind is probably the smallest Crowd GPS tag. It was on indiegogo and raised 931.870 US-Dollar.

It works with Android and iOS, as long as you have Bluetooth 4.0.

Specs

Technology: Bluetooth 4.0 (Bluetooth Low Energy)
Range: Approximate 30m with line of sight.
Battery: Lasts up-to 1 year based on 30 minutes per day average use.
Battery Type: CR2016 watch battery. Battery is replaceable
Cost: A 10 pack costs 200 US-Dollar.
Dimensions
- Shape: Cylindric
- Diameter: 2.4cm diameter
- Thickness: 4mm
- Weight: 4.5g

SelectaDNA

Bicycles

Bicycles sometimes have a number on the frame. In German, it is called Fahrradrahmennummer. In Germany, you can sometimes go to the police and they will add such a number to your bike. In case it gets stolen and the police finds it, they can look at their records and give it back to you. Eventually. I’m not sure how good this works. But it doesn’t cost anything, so why not?

Another way is uglification. One way to do this is using these stickers.

Another way is putting lots of reflecting stuff on it. That has the advantage of giving you extra security as drivers can see you more easily.

USB-Sticks

I always add a README.txt to the USB stick with the following content:

## German
Dieser USB-Stick gehört Martin Thoma.

Wenn er verloren wurde, wäre es toll, wenn Sie eine E-Mail an
info@martin-thoma.de schreiben würden.
--------------------------------------------------------------
## English
This USB stick belongs to Martin Thoma.

If it was lost, please send an e-mail to info@martin-thoma.de.

Luggage

I always leave a note in my luggage with the following content:

Martin Thoma
Parkstr. 17
76131 Karlsruhe
Tel.: XXXX
E-Mail: info@martin-thoma.de

House

Locks and keys… but you can lose your keys or forget them inside.

When you lose your keys, you have to replace all locks. That’s a problem with traditional keys. But I could imagine that a good keycard lock does not have this flaw. You “simply” tell the system that a card was lost and replace the cheap card - not the expensive lock. The card should only contain a long number. If the number is one of the allowed numbers, one can enter. If not, the system locks for 2 seconds. That should effectively prevent brute force.

Smart Phones

Android device manager

Packaging with Python

info@martin-thoma.de (Martin Thoma) — Mon, 29 Dec 2014 21:05:00 +0100

This is a quick article I had for quite a while as a draft. It might not be finished or have other problems, but I still want to share it. I wrote this when I did not know much about packaging.

The following article is a wrap-up of the talk Python Packages from Daniel Hepper given at a German PyCon 2013.

distutils have very limited functionality. This is the reason why you should use setuptools. But ‘distribute’ is compatible to setuptools.

PyPi is the Python Package Index. They distribute packages in form of “eggs”. You can install them with easy_install or with pip.

virtualenv / virtualenvwrapper

Package Management Tools

Distutils

Distutils is part of the standard library. When you run

python setup.py install

then distutils is running.

Major disadvantages of distutils are:

No Meta-data:
- no deinstallation
- no dependencies
No Package listing (so you can’t automatically search pypi)

Setuptools

Setuptools are startet when you install a package with easy_install:

easy_install package

Setuptools are an extension for distutils. Setuptools offers dependency management. With setuptools, so called ‘egg files’ were introduced. Those files are comparable to jar files in Java.

Distribute

Distribute was a fork of setuptools that got merged back to setuptools. So don’t use distribute, but use setuptools.

Installer

PIP

PIP is short for ‘PIP installs Python’. It can only install files from sources; so it does not support egg files.

PIP commands are

install
uninstall
freeze
search
bundle
unzip
zip
wheel
help

Environments

It might be the case that you have to have a Django 1.4 and a Django 1.5 project. In that case, you need different environments.

Virtualenv

$ pip freeze
$ virtualenv my_env
$ source my_env/bin/activate

Virtualenvwrapper

Creating packages

A project could have this structure:

$ tree
.
├── LICENSE
├── MANIFEST
├── pyconde2013news
│   └── __init__.py
├── README
└── setup.py

With this setup.py:

from distutils.core import setup
setup(name="pycon2013news", version="0.1", py_modules=["pycon2013news"],)

This can directly be registered on PyPi:

python setup.py register

But a problem of this code is that it does not show the dependencies. So you should rather use setuptools for your setup.py:

from setuptools import setup
setup(name="pycon2013news",
      version="0.1",
      py_modules=["pycon2013news"],
      description="Reads latest news headlines from the PyCon.DE 2013 website")
      author="Daniel Hepper",
      author_email="daniel@butfriendly.com",
      url="https://github.com/dhepper/pyconde2013news",
      install_requires=[
          "beautifulsoup4==4.3.2",
          "requests==2.0.0"
      ]

When you add scripts argument to setup, you can later execute those. Entry points are also interesting.

Additional information

http://guide.python-distribute.org/contributing.html

Chatbots

info@martin-thoma.de (Martin Thoma) — Mon, 29 Dec 2014 21:05:00 +0100

The Turing test is an experiment that was proposed by Alan Turing to test the ability of an artificial intelligence to emulate intelligent behaviour. It works like this:

A human judge gets to talk over a chat client with either a human he doesn’t know or a chat robot. If the judge can distinguish the human from the chat robot, the chat robot isn’t intelligent.

(There is a lot more to say about the Turing test. If you’re really interested, you should read the wikipedia article.)

Let’s see how long it takes for me to say that I’m either talking to a very dumb person or to a computer for various chat robots

A.L.I.C.E.

See alice.pandorabots.com.

judge: hi ALICE: Hi there! judge: How are you? ALICE: I am fine, thank you. judge: What are your hobbies? ALICE: I like robots, computers, and chatting online. judge: So we do ☺ ALICE: Really.. I’m glad you find this amusing. judge: What is 7+3? ALICE: Use a calculator! judge: What was my last question? ALICE: Is this a trick question? judge: No. ALICE: It seems like a trick to me. judge: Could you simply answer it? ALICE: Please rephrase your question, judge. judge: Bye ALICE: Bye bye.

Check	Difficulty	How the Chatbot performed	Stars
Greetings	Easy	He greated just as I’ve expected it.	5 / 5
Smalltalk	Average	He did not keep the talk running and he always answer the same way to smilies.	3 / 5
Tasks	Average	He avoided the task. Which is ok.	4 / 5
Back reference	Hard	Asking for a trick question and for rephrasing might be a good way to get the human to give the context in one chat message. Very good!	3 / 5

Cleverbot

See cleverbot.com.

judge: hi. Cleverbot: How are you? judge: I’m fine. How are you? Cleverbot: Good. judge: What are your hobbies? Cleverbot: Talking to query bots, you? judge: Hehe, very good. I like reading and going to the cinema. Cleverbot: What books do you like? judge: The sword of truth, Lord of the rings, The Dresden files. Cleverbot: Ok… judge: What is 7+3? Cleverbot: What is what? judge: What is the capital of the U.S.? Cleverbot: I don’t know. judge: Do you know any city? Cleverbot: No. judge: Where do you live? Cleverbot: Western Massachusetts. judge: That’s the state. And in which city do you live? Cleverbot: I will not tell. judge: Ok. Can you tell me what my last question was? Cleverbot: How can I get you to trust me. judge: Bye. Cleverbot: Ok, bye.

Check	Difficulty	How the Chatbot performed	Stars
Greetings	Easy	He didn’t greet at the beginning.	2 / 5
Smalltalk	Average	Quite ok.	4 / 5
Tasks	Average		2 / 5
Back reference	Hard		1 / 5

This one tried to lead the conversation. I think that might be a good way to emulate a human. Either don’t talk or keep talking all the time so that you (your chatbot) know the context.

Cronjobs

info@martin-thoma.de (Martin Thoma) — Mon, 29 Dec 2014 21:00:00 +0100

Cron is a very usefull service in Linux systems that allows periodical execution of programs.

Terms

The program that runs other programs periodically is called “cron”. The configuration file, that tells cron which programs should be run periodically is called “crontab” and located in /etc/crontab. One line in that file is called “cronjob”. But “crontab” is also a command that lets you add cronjobs.

Example

The following entry in crontab means that uamt.py is executed every day at 3:30 am:

30 3 * * * /home/mthoma/uamt.py

You can check if everything went fine with

mthoma@i13srv30:~> crontab -l
# DO NOT EDIT THIS FILE - edit the master and reinstall.
# (/tmp/crontab.DEZBGc installed on Sat Jan  4 14:35:03 2014)
# (Cronie version 4.2)
30 3 * * * /home/mthoma/uamt.py

Resources

HowTo: Add Jobs To cron Under Linux or UNIX?

SVN am KIT

info@martin-thoma.de (Martin Thoma) — Mon, 29 Dec 2014 20:57:00 +0100

This is a quick article I had for quite a while as a draft. It might not be finished or have other problems, but I still want to share it.

Jeder Student mit einem ATIS-Account kann hier einen SVN-Zugang beantragen. Ich werde nun kurz erklären, wie man ihn mit der Konsole benutzt.

Import

Syntax (Hilfe):

svn import [PATH] URL

Beispiel:

svn import -m "Initial import of the BankAccountReader project." /home/swt-user/BankAccountReader https://svnserver.informatik.kit.edu/stud/svn/s_thoma/trunk/BankAccountReader

Checkout

Syntax (Hilfe):

svn checkout URL[@REV]... [PATH]

Beispiel:

svn co https://svnserver.informatik.kit.edu/stud/svn/s_thoma/trunk --username s_thoma

Copy, Move, Delete

Copy (Hilfe):

svn copy SRC[@REV]... DST

Beispiel:

svn copy mySoureFile.java folder/myDestFile.java

Move (Hilfe):

svn move SRC... DST

Delete (Help):

svn delete PATH...

Für delete gibt es einige synonyme Befehle:

svn rm myFile.java

Anders als der standard Linux rm-Befehl ist der von SVN rekursiv!

status

Syntax (Hilfe):

svn status [PATH...]

Beispiel:

svn status

D       src
D       src/Tests.in
D       src/Tests.out
D       src/banking
D       src/banking/Status.java
D       src/banking/AccountNumber.java
D       src/banking/Shell.java
D       src/banking/Terminal.java
D       src/svn-commit.tmp
A  +    BankAccountReader
D       BankAccountReader/svn-commit.tmp

commit

Syntax (Hilfe):

svn commit [PATH...]

Beispiel:

svn commit -m "restructuring directory structure"
Adding         BankAccountReader
Deleting       BankAccountReader/svn-commit.tmp
Deleting       src

Committed revision 3.

Siehe auch

Subversion complete reference

Subversion

info@martin-thoma.de (Martin Thoma) — Mon, 29 Dec 2014 20:57:00 +0100

This is a quick article I had for quite a while as a draft. It might not be finished or have other problems, but I still want to share it.

Subversion or Apache Subversion or short svn is a is a software versioning and a revision control system. This allows you to step back to any point of the software and to develop simultaniously on the same project (but not on the same file). You have to make sure that no dependencies are broken, of course.

Here is a little cheat sheet how to use SVN. I will demonstrate some on my Google Code OpenSource project Community Chess.

All commands are executed while I’m in my local working copy of the repository.

Get the repository

svn checkout https://community-chess.googlecode.com/svn/trunk/ community-chess --username themoosemind@googlemail.com

Update to the latest version

svn update

Check for changes

If you only want to check which files were modified, added or deleted you can execute

svn status

This will show your local changes.

File actions

The actions are simmilar to the console commands, but you have to add svn:

Copy

svn cp myFile.php copiedFile.php

Delete

svn rm myFile.php

Rename / Move

svn mv myFile.php folder/myNewFile.php

Add file

If you created a file and want to submit it with the next commit, just do

svn add myFile.php

Commit the latest changes

Check your changes

First you should try svn status. What files did you change? Do you really want to upload those changes?

If svn status gives you an exclamation mark (!), you might have deleted a file which you wanted to add before. No problem. Just make

svn rever /path/to/your/file.php

I'm fine: Upload it!

This is the command you use, if you want to send the changes you made on your working copy to the repository:

svn commit -m "Moved some functions to additional.inc.php to keep the project more flexible; Much work for tournament implementation done; Some Warnings fixed" --username themoosemind@gmail.com

Watch changes

General

See the last three changes:

svn log -v --limit=3

------------------------------------------------------------------------
r118 | themoosemind@gmail.com | 2011-09-26 23:18:42 +0200 (Mo, 26. Sep 2011) | 1 Zeile
Geänderte Pfade:
   M /trunk/clients/Java/ChessClient.class
   M /trunk/clients/Java/ChessClient.java

small changes on the Java Client
------------------------------------------------------------------------
r116 | themoosemind@gmail.com | 2011-09-26 22:53:01 +0200 (Mo, 26. Sep 2011) | 1 Zeile
Geänderte Pfade:
   A /trunk/clients/Java
   A /trunk/clients/Java/ChessClient.class
   A /trunk/clients/Java/ChessClient.java
   A /trunk/install/phpBB3
   A /trunk/install/phpBB3/login.wrapper.php (von /trunk/install/phpbb3.login.php:100)
   A /trunk/install/phpBB3/phpbb3.integration.txt (von /trunk/install/phpbb3.integration.txt:113)
   A /trunk/install/phpBB3/phpbb3.php (von /trunk/install/phpbb3.php:100)
   A /trunk/install/phpBB3/phpbb3.sql (von /trunk/install/phpbb3.sql:103)
   A /trunk/install/phpBB3/wrapper.inc.php (von /trunk/install/phpbb3.wrapper.inc.php:113)
   D /trunk/install/phpbb3.integration.txt
   D /trunk/install/phpbb3.login.php
   D /trunk/install/phpbb3.php
   D /trunk/install/phpbb3.sql
   D /trunk/install/phpbb3.wrapper.inc.php

Made integration instructions for phpBB easier; added Java-Client
------------------------------------------------------------------------
r113 | themoosemind@gmail.com | 2011-09-26 08:29:15 +0200 (Mo, 26. Sep 2011) | 1 Zeile
Geänderte Pfade:
   M /trunk/index.php
   M /trunk/install/chess.sql
   M /trunk/install/phpbb3.integration.txt
   M /trunk/install/phpbb3.wrapper.inc.php
   M /trunk/my_software.php
   M /trunk/wrapper.inc.php

Fixed issue 3 and some more phpBB-Bugs; more detailed integration instructions;Renamed GAMES_THREEFOLD_REPETITION_TABLE
------------------------------------------------------------------------

One specific file

I don’t want to get that much output, so I don’t apply -v this time:

svn log index.php --limit=3

------------------------------------------------------------------------
r127 | themoosemind@gmail.com | 2011-10-04 00:09:59 +0200 (Di, 04. Okt 2011) | 1 Zeile

Moved some functions to additional.inc.php to keep the project more flexible; Much work for tournament implementation done; Some Warnings fixed
------------------------------------------------------------------------
r113 | themoosemind@gmail.com | 2011-09-26 08:29:15 +0200 (Mo, 26. Sep 2011) | 1 Zeile

Fixed issue 3 and some more phpBB-Bugs; more detailed integration instructions;Renamed GAMES_THREEFOLD_REPETITION_TABLE
------------------------------------------------------------------------
r71 | themoosemind@gmail.com | 2011-08-26 23:13:37 +0200 (Fr, 26. Aug 2011) | 1 Zeile

completed MVC with Vemplator; controlled every php file with phpcs
------------------------------------------------------------------------

Scherzfragen

info@martin-thoma.de (Martin Thoma) — Mon, 29 Dec 2014 20:56:00 +0100

This is a quick article I had for quite a while as a draft. It might not be finished or have other problems, but I still want to share it.

Um die Antworten der folgenden Scherzfrangen zu erhalten müsst ihr einfach den schwarzen Text markieren.

Tierkonferenz

Wie bekommt man den Elefanten in den Kühlschrank? Kühlschranktür auf, Elefant rein, Kühlschranktür zu.

Wie bekommt man die Giraffe in den Kühlschrank? Kühlschranktür auf, Elefant raus, Giraffe rein, Kühlschranktür zu.

Wichtige Tierkonferenz: Welches Tier fehlt? Die Giraffe, die ist noch im Kühlschrank.

Du kommst im Urwald zu einem Fluss mit gefährlichen Krokodilen. Keine Brücke, kein Boot…nichts. Wie kommst du hinüber? Schwimmen - Die Krokodile sind auf der Konferenz.

Wie merkt man, dass ein Elefant im Kühlschrank war? An den Fußstapfen in der Butter.

Warum darf man nach 24 Uhr nicht mehr in den Urwald? Weil dann die elefanten Fallschirm springen.

Warum haben Krokodile so platte Nasen? Weil sie nach 24 Uhr im Urwald waren.

Autokollision

Was sagt ein Hund kurz bevor er von einem Auto angefahren wird? hilf…

Und ein Pferd? hilf…

Und ein Elefant? Komm nur.

Busfahrer

Du bist der Busfahrer. Der Bus startet den Motor und fährt los. Es steigen fünf Leute ein. Er fährt weiter und drei steigen aus. Weiter geht es, und sieben steigen ein, und fünf aus. Der Bus hält und alle müssen aussteigen weil jetzt Endhaltestation ist.

Frage: Wie alt ist der Busfahrer? DU bist der Busfahrer!

Schnell antworten

Was ist schwerer, ein Kilo Federn, oder ein Kilo Blei? Es ist beides gleichschwer, ein kg ist ein kg.

Auf einer Stange sitzen 10 Tauben. Ein Jäger schießt eine Taube ab. Wie viele sitzen noch da? Es sitzt keine Taube mehr auf der Stange. Wenn eine erschossen wird fliegen die anderen weg.

Du hast ein Streichholz und kommst in einen leeren kalten Raum, in dem du nichts weiter als eine Petroliumlampe, einen Kamin und einen Ölofen vorfindest. Was zündest du zuerst an? Bevor du irgendetwas anzünden kannst musst du zuerst das Streichholz anzünden.

Welche Farbe hat Papier? Weiß.

Welche Farbe hat die Heizung? Weiß.

Welche Farbe hat die Wand? Weiß.

Was trinken Kühe? Wasser.

Ein Flugzeug stürzt auf der Grenze von Frankreich zu Deutschland ab. Wo werden die Überlebenden begraben? Überlebende werden nicht begraben.

Scherzfragen

Wie kann man Postbote ohne O schreiben? Briefträger.

Was macht 999 mal “Tick” und einmal “Tack”? Ein Tausendfüßler mit einem Holzfuß.

Was schwimmt und fängt mit B an? Brett.

Was schwimmt und fängt mit N an? Noch ein Brett.

Was schwimmt und fängt mit S an? Schon wieder ein Brett.

Was ist braun, groß, lebt 10 Meter unter der Erde und frisst Steine? Der große, braune, unterirdische Steinfresser.

Was ist braun, lebt 10 Meter unter der Erde und Frisst Sand? Die Oma vom großen, braunen, unterirdischen Steinfresser.

Was sagt Tarzan wenn er ne Herde Elefanten durch den Wald laufen sieht? Guck mal, eine Herde Elefanten! Was sagt Tarzan wenn er eine Herde Elefanten mit Sonnenbrille durch den Wald laufen sieht? Nichts, denn er erkennt sie nicht. Was sagt Tarzan wenn er ne Herde Giraffen durch den Wald laufen sieht? Nochmal fall ich auf euer Verkleidungsspiel nicht rein!

Wenn man ein Loch von Berlin aus quer durch die Erde bohren und einen Stein reinwerfen würde, wie weit würde er fallen? 10 Meter, dann wird er vom großen, braunen unterirdischen Steinfresser aufgefressen.

Weitere

Wer reist ständig kostenlos um die Welt??? Der Mond.

Java Exceptions

info@martin-thoma.de (Martin Thoma) — Mon, 29 Dec 2014 20:39:00 +0100

This is a quick article I had for quite a while as a draft. It might not be finished or have other problems, but I still want to share it.

When to use Errors and Exceptions

An Error is a subclass of Throwable that indicates serious problems that a reasonable application should not try to catch. Most such errors are abnormal conditions.

(Source: Javadoc)

How to throw an exception

public void myMethod( String s ) {
  if (!isStringValid(s)) {
    throw new IllegalArgumentException("This string is not valid!");
  }
}

Common Exceptions

IllegalArgumentException: One argument of the current method hasn’t the form it should have.
IllegalStateException: The current object is in the wrong state.
NullPointerException: A Null-Pointer was given, but it should have been an object.

A long list of Exceptions is on Hai’s Blog.

Famous Software Bugs

info@martin-thoma.de (Martin Thoma) — Mon, 29 Dec 2014 20:32:00 +0100

This is a quick article I had for quite a while as a draft. It might not be finished or have other problems, but I still want to share it.

Mars Climate Oribiter

Type of Bug: Bad specification
Description: The flight system software on the Mars Climate Orbiter was written to calculate thruster performance using the metric unit Newtons (N), while the ground crew was entering course correction and thruster data using the Imperial measure Pound-force (lbf).
Outcome: The cost of the mission was $327.6 million total for both orbiter and lander, $ 193.1 million for spacecraft development, $91.7 million for launching it, and $ 42.8 million for mission operations.
Source: Wikipedia

Ariane V88

The Ariane V88 exploded 40 seconds after its start.

Type of Bug: The software was written for another type of hardware.
Description: A 64 Bit floating point number was converted into a 16 bit integer in the “inertial reference system” → Overflow → the rocket got into a tilted position and destroyed itself for security reasons. The interesting part is, that this program wasn’t even needed for the flight! It had been developed for the Ariane 4.
Outcome: 290 Million Euro destroyed

Other

Do you know more?

Replicators

info@martin-thoma.de (Martin Thoma) — Mon, 29 Dec 2014 20:25:00 +0100

This is a quick article I had for quite a while as a draft. It might not be finished or have other problems, but I still want to share it.

What would you need to explore space? There are some ideas in science fiction like interstellar arks and near lightspeed travel. But one concept might be easier to realize: Self-replicating, autonomous machines.

The Problem of Interstellar Arks

Interstellar arks are space ships that are of enourmous size. They are big enough to have enough humans on it so that those humans can have childens over an limitles time without getting into trouble because of incest. Traill et al. [2007] suggests that you need about 4000 individuals for a minimal viable population.

Those 4000 people would need food, air, personal space. You would need resources to repair the ship and explore planets. Medicine. You would need fuel. That would be a lot of stuff.

Bringing stuff from earth to space is expensive.

And those people would eventually be trapped on that ship for many, many generations before they might find another habitable planet. You might eventually need to do terraforming. I can’t even imagine the number of resources that would be needed for that.

The next problem is that you would want the explorers to keep contact with earth. Those signals would either have to be highligh redundant (see error correcting codes) or they might get received only partially. So eventually you would like to get relay stations that catch up the signal, correct errors (or request a new send of the broken part) and send it again.

An ark would have to carry everything for that. That’s a problem because sending so much stuff from earth to space is expensive.

The alternative: Replicating Machines

An alternative might be replicating machines. Machines don’t need personal space. They don’t need air. They don’t need food or medicine. They don’t run into psychical problems.

But they do need energy. They need resources. And they either need to be really intelligent or they need to have clear instructions.

I a working minimal space exploring construction kit (let’s call it MISECK, because that’s much shorter than ‘minimal space exploring construction set’) should be able to do the following:

Construct other MISECKs from resources that can be found in space (e.g. on asteroids, moons or planets). The plural was intended. Every MISECK would have to be able to produce at least two other MISECKs.
Build communication satelites: That communication satelite would act as a relay station. It talks with other satelites, receives messages and sends messages. It is able to check if the received message has errors and request a new message in case of errors. They should also store as much information as possible. This way, one could eventually contact them and get information that might otherwise get lost.
Build exploration satelites which can gather information about a planet. This information should be send to the communication satelites and finally back to earth.
Build mining and construction robots that are able to mine all resources needed to construct MISECKS. That will include melting ore and constructing fabrics which are able to produce computer chips.
Build power plants. Those might be solar cells, fusion/fission reactors or something completely different. But something has to provide the energy to make it possible to run all those machines and to travel enormous distances in space.
Explore and coordinate: This is a collaborative planning task. MISECKs have to know where other MISECKs are. If one MISECK gets destroyed by anything, there might still be something that is worth exploring.

How to draw speech bubbles

info@martin-thoma.de (Martin Thoma) — Mon, 29 Dec 2014 20:20:00 +0100

This is a quick article I had for quite a while as a draft. It might not be finished or have other problems, but I still want to share it.

The nice icon is from Wikipedia Commons and part of the Crystal Clear project by Everaldo Coelho.

Preparation

Install inkscape.
Get an image where you want to put the speech bubble.
Add text with F2

Now it should looke like this:

Preparation

Rectangle

Now you have to put a rectangle (F2) or an ellipses (F5) around your text. The rectangle should be placed below the text (Page down).

To make positioning easier, group the text and the surrounding box:

Mark box by clicking on it
Shift + Click on the text
Group with Ctrl + G

Now it should looke like this:

Preparation

Indicator

To add the “indicator” that shows who spoke, press Shift + F6 and add a triangle that overlaps with the box:

Preparation

Then press F2 to modify the path by nodes. Add some controll nodes in between by double-clicking on the path in between:

Preparation

Then delete them. Now you have those round curves that can be manipulated with the little circles:

Preparation

Logo Design Tournament

info@martin-thoma.de (Martin Thoma) — Mon, 29 Dec 2014 19:54:00 +0100

The website logotournament.com offers a service for designers and owners of companies. You can define how much money you would like to spend for a new logo-set and how this logo should look like.

Logo Tournament Website

Just take a look at the contest brief of Move Ahead. I’ve made a screenshot of their Website. They use the typical Joomla-Icon. Their Logo isn’t very good.

Proposed Logos on Logo Tournament

Now take a look at the best submitted logos. Great, aren’t they? At the moment, 105 logos have been submitted for $375. The company can decide which ones they like and rank them. They pay only for one logo set, but they can choose from 105 logos.

Move Ahead Website

Note: This is a quick post from one of my old blogs.

How mathematics could change our future

info@martin-thoma.de (Martin Thoma) — Fri, 26 Dec 2014 00:56:00 +0100

Can you think of a solution for a problem that concerns the whole or most of humanity and could be solved for the next ten years or even forever with less than 1 million dollars?

This was the title of a question I’ve recently found on Quora. I think this question is very interesting and I would like to share my thoughts about it.

I think the answer is mathematics and computer science. Or lets rather say algorithms.

There are multiple problems in mathematics and computer science which could have a severe impact on some areas which are important for the worlds economy. As we live in a globalized world, this will affect most of humanity. As algorithms don’t cost anything except for developing cost, it is certainly below 1 million dollars. In principle, everybody could come up with such algorithms.

P vs NP

The P versus NP problem is one of the Millennium Prize Problems. Stated extremely simple, it asks whether every problem whose solution can be quickly verified by a computer can also be quickly solved by a computer. If somebody found an algorithm which solves one problem in the class NPC in fast, it will eventually boost many problems or allow us to solve problems which were not solvable before. Especially optimization problems.

Optimization problems

There could be other algorithms which boost optimization problems, but do not help for the P vs. NP problem.

Optimization is about allocating just the right amount of resources for a couple of goals to get the highest outcome.

Fully homomorphic encryption

Finding efficient solutions for the problem that we don’t want cloud computing providers to know our data, but we want them to use their efficient data centers to run calculations on them. This would boost cloud computing and lead to a more efficient use of computational resources.

A.I.

The development of a strong A.I. could lead to a technological singularity. This would be a boost for technology incomparable to anything before.

However, even advancements in A.I. which are not strong A.I.s will help.

They can lead to many new products where humans cannot cope with the data volume. One example would be medicine. IBM’s Watson is one first step to help doctors skip through the possible thousands of diagnoses and finding the most relevant and recent papers which could fit the problems of a patient. New diagnoses could be developed, the computer could help to create better (more accurate) tests for diseases. It could help to track the spread of diseases, predict it and thus help to contain it.

Parallelization and Compilers

Developing a tool which automatically parallelizes sequential code could make all applications go faster. The same is true for advances in compiler technology. This could lead to less energy consumption and faster devices. As most people have contact to computers nowadays, this would affect everybody.

Can you think of other areas where algorithms have a strong impact?

Shortfilms, Part III

info@martin-thoma.de (Martin Thoma) — Thu, 11 Dec 2014 22:54:00 +0100

Here is the second part "Shortfilms, Part II".

I’ve collected quite a few YouTube videos in my “watch later” list. Most of them were short movies which I wanted to share. Here you are!

Azarkant

The Raven

The Archiver

Hybrids

SEED

REWIND

Not a short movie … but interesting nevertheless ☺

A Darwinian Future

OMEGA

Paradox

One Rat Short

The Chase

Devils, Angels & Dating

True Skin

The Silent City

The Escape

Sam

Giant Robot vs Monster

WITCH

ATTENTION: Very loud at the beginning!

Rosa

Natalis Teaser

ABE

Freaky … and scary (Psycho Horror)

SpyFox

When you KILL Little Creatures on the road, there are consequences!

Devil Claim

BEAR ‘N’ WASTELAND

BLIK

LEGACY

TOLERANTIA

ARK

Copia

Meet Buck

Egghunt

Play Audio

info@martin-thoma.de (Martin Thoma) — Sun, 07 Dec 2014 00:31:00 +0100

Our senses have a limited capability to distinguish signals. This might not be surpristing. However, we also have a limited capability to tell in which order signals arrived at our sensory organs.

As I learned this, I wanted to check it. I wrote a little JavaScript / HTML5 page where you can check it. You can adjust the time in which two sounds are played. The clicking noise is played on the left and the right speaker in random order. A textfield below shows which speaker played the sound first.

You should use headphones for this demo.

According to the KIT lecture “Mensch-Maschine-Wechselwirkung in der Anthropomatik (Vorlesung Universität Karlsruhe im KIT, WS 2013/14, J. Geisler)”, the time resolution of the human ear is between 2 and 5 ms. This means you will not be able to distinguish the two signlas for less than 2ms. But you will need the signals to be 30-40ms apart to tell which one played first.

I rather need 10ms, but this might also be caused by hardware problems. I doubt that JavaScripts timeout is accurate enough.

Audio Test

Play sound ms

Which was first

The Impact of strong AIs

info@martin-thoma.de (Martin Thoma) — Tue, 02 Dec 2014 15:18:00 +0100

An artificial intelligence (AI) is a computer program which acts - according to its developers - intelligent. That could be anything. Typical examples are route finding algorithms, chess programs, ego shooter computer opponents and classifiers. The last category of AIs (classifiers) is huge and includes programs which try to find out what you were writing (see my bachelor’s thesis), try to figure out who is on an image (face recognition) or what was spoken (automatic speech recognition).

Strong and Weak AIs

Science fiction literature distinguishes two kinds of AIs: Weak AIs and strong AIs.

Currently, we only know weak AIs. They can do incredible things (see A.I. in Computer Games and Awesome Robots), but that is nothing compared to a stong AI. Strong AIs are capable of adapting to completely new tasks. They can do creative work. In other words, they can do any task any human could do. They can do research and compose the most beautiful art. As they are machines which will be build by somebody (hence the term ‘artificial’), they are better understood than our biological brains. Strong AIs will be able to understand how they work themselfes and very likely be able to improve themselfes.

This is the point where a technological singularity happens. The machines start developing faster than any human can comprehend how they improve. At some point they will even develop faster than humankind together can understand.

Now that you know the context, I would like to share an answer to the following question I’ve recently seen:

What would change with strong AIs?

What happens when machines take all of our jobs including creative/research/artistic ones?

Something like that would not happen instantly. It is a gradual process. Many answers say something like nothing would be scarce. Well, I think that is wrong for two reasons:

Scarcity of resources: Oil, gold and diamonds are expensive. But they are not (only) expensive because it takes a high amount of work to get them, but also because the number of diamonds on earth is limited. More or cheaper “work hours” like much better AIs combined with much better robots would not solve that problem. Oh, and please don’t forget energy!
Distribution: I think we produce enough food so that nobody on earth would have to starve. We certainly have enough clean water for everybody. Why do so many people still starve and don’t have access to clean water? Because the distribution is not equal. Europe and the US make use of much more resources per person than Africa does. The richest 1% of the US make use of MUCH more resources than the bottom 30% (I guess the numbers are more extreme). Who do you think would own the robots that take the jobs? Who would profit from this much cheaper work hours? I think AIs that would in principle be able to solve any problem a human could solve (e.g. creating art and conducting research) would cause serious social problems if we don’t adapt to the new situation. Such AIs would have the potential for a much worse world than we currently live in.

However, with the right politics, it could vastly improve our world. Having the insight how such AIs work, we could make them decide ultimatively unbiased for a greater good. They could be used to arbitrate a dispute as a neutral, intelligent instance. They could accelerate research. They could help us to understand ourselves.

Job market

What is the effect of technological singularity in job market?

In a ideal world, everybody would only do the job he or she wants to do. We would eventually work less, but I think we would still work. Humans are resources and as such they will always be valuable in any economy.

Another, darker, scenario is that AIs would gradually remove whole industries, starting with simple ones. Taxi and truck drivers are not necessary. They could step-by-step be replaced by AIs. However, no new jobs would be created for those people. They would have to get aid by the government. But as they would get less money, the economy would focus on the people who have money. That would be the people who own the AIs. At some point people would realize that they will never be able to get a new job. Even worse, their children will never be able to get a job. Extreme poverty would rise as the state gets less taxes (as less goods are consumed, because people have less money). The AIs would predict how every single person would most likely act. How could they do so? Well, you have a smartphone. Your conversations on WhatsApp, Twitter, Facebook, Gmail, … get tracked and automatically analyzed. You can be predicted to a certain degree. You can be influenced by personalized advertising. A really clever AI will make itself able to act in any possible scenario, replicate itself and make itself less dependent. In this dark scenario people would eventually start at some point to try to get more money from the people controlling the AIs, but how do you force them to do so? The police might also be replaced by AIs.

My guess is that the reality would be somewhere in between. A couple of super-rich people and the rest giving them massages.

I’ve just described that AIs might cause serious social problems. However, I think we can solve those problems by making sure that income inequality cannot get too high. This means there should be very high and effective inheritance tax as well as a taxation system that prevents people from getting too rich / people too poor. The most extrem action to prevent too poor people is an unconditional income, the most extrem action to prevent people getting too rich is an upper limit on what somebody could have.

I think an unconditional income would be a good thing, but its hard to tell how high that should be.

The easiest way to prevent people from getting too rich is adding a tax system that adjusts to income:

Your first 0 - 2000 Euro / month don’t get taxed at all
Your next 2001 - 3000 Euro / month get taxed with 0% + (100%/2) = 50%
Your next 3001 - 4000 Euro / month get taxed with 50% + (50%/2) = 75%
Your next 4001 - 5000 Euro / month get taxed with 75% + (25%/2) = 87.5%
…

You can (and should - I think my numbers are not well-chosen!) argue about the exact numbers, but I guess you get what I mean. The tax should never be 100%, thus leaving the possibility to get more money. But the difficulty to do so should increase. This effectively prevents that some people get too rich and hence resulting in a very instable system.

It would be important to do so before anybody develops a strong AI.

Books (all fiction)

“Zero” by Marc Elsberg: People get controlled by life improvement apps in a very indirect way.
Out-Series by Andreas Eschbach: A device was developed, that lets people connect their brains. A new form of conciousness develops from that.
Brave new world by Aldous Huxley: People get distracted from issues by consuming many goods.

Movies (all fiction)

Transcendence: The mind of one person gets transformed in a computer.
I, Robot: An AI gets developed and very powerful humanoid robots get controlled by it.

Talks

The rise of the new global super-rich: Technology is advancing in leaps and bounds — and so is economic inequality, says writer Chrystia Freeland. In an impassioned talk, she charts the rise of a new class of plutocrats (those who are extremely powerful because they are extremely wealthy), and suggests that globalization and new technology are actually fueling, rather than closing, the global income gap. Freeland lays out three problems with plutocracy … and one glimmer of hope.

And a coulple of talks by people who are not active within AI / ML research themselves:

Nick Bostrom: “Superintelligence” - Strong AI is inevitable; we should set the initial conditions up the right way
Sam Harris and Joe Rogan talking about artificial intelligence
Max Tegmark and Nick Bostrom speak to the UN about the threat of AI
- UNICRI

Articles

9 crazy things that could happen after the singularity, when robots become smarter than humans

Universal Rating System

info@martin-thoma.de (Martin Thoma) — Sat, 22 Nov 2014 17:19:00 +0100

A key feature of Amazon is its rating system. Rating systems are relatively straight-forward to build, but they need much data and get better with more data. I think Amazon builds a monopoly that way. However, I also think we could build an Universal Rating System (URS).

Just as important as pure rating are reviews.

What would an URS need?

Directly neccessary are

Identification of people: The same person should not be able to write multiple reviews or make multiple ratings for the same product. It is not important to tell which online-identity belongs to which person. In fact, I think it would be good if it was possible to keep users anonym while making sure that everybody has only one online-identity.
Identification of products: Users have to be able to find products and they have to find reviews to that product. So some kind of product identification number is necessary.
Ratings for reviews
Uploading photographs
Product description: This should be objective.

Indirectly necessary are

Recommendations: If users see that they get good recommendations from an independant source they might use the system. If they see that those recommendations imporve with more reviews and more ratings, they add more of them. This way the system gets better.
Possibility to compare products: Examples are phonearena.com, ark.intel.com and others. But they are always limited to some products.
Search: Users should be able to search for a product by specifying it. This could be a price range, a category, a review average, a minimum number of reviews, …
Easy login: Something like OpenID that makes it easy for users to create an account and sign in.
*Right incentives: Having a great community is much about giving the right incentives. Badages and Karma are a great way to do so as shown with StackExchange.

Non-software parts

Advertisment: People have to get informed about such a system.
Organization: The software should be OpenSource, the data should be published (as far as possible without breaking privacy) and the organization which runs the software should be a nonprofit organization. The reason is simply that it prevents abuse.
Product registration: Companies should have the possibility to register their products to get an identification number.
Project name: A good name is important. Any ideas?

Existing Projects

Does anybody know if such a project already exists?

If there is no such project by now: Is anybody interested in working on it?

Nyquist–Shannon sampling theorem

info@martin-thoma.de (Martin Thoma) — Tue, 18 Nov 2014 17:41:00 +0100

The Nyquist–Shannon sampling theorem states that you have to sample more than twice the highest frequency. If you sample less often, you will get aliasing.

The following videos show what aliasing is:

Formale Systeme Klausur

info@martin-thoma.de (Martin Thoma) — Sun, 16 Nov 2014 17:05:00 +0100

Dieser Artikel beschäftigt sich mit der Vorlesung „Formale Systeme“ am KIT. Er dient als Prüfungsvorbereitung. Ich habe die Vorlesungen bei Herrn Prof. Dr. Beckert im Wintersemester 2014/2015 gehört.

Behandelter Stoff

Folien

Folien- satz	Inhalt
01	Organisatorisches
02	Aussagenlogik: Modellierung von Sudoku; 8-Damen-Problem; Allgemeine Syntax und Semantik sowie Grundbegriffe, Tautologien und Sätze; Basis; Erfüllbarkeit; Allgemeingültigkeit; $M \models A$
03	Aussagenlogik: Craig-Interpolation
04	Aussagenlogik: Normalformen (KNF, DNF, KKNF)
05	Binary Decision Diagrams: (normierte) Shannon-Formeln, sh-Operator, Shannon-Graph, Reduzierte Shannon-Graphen
06	SAT; Satz von Cook; Horn-Formeln; DPLL
07	Prädikatenlogik: Syntax (PL1); JML; (Kollisionsfreie) Substitutionen; Unifikation und der Algorithmus von Robinson; Unifikationstheorem
08	Pradikatenlogik: Semantik; Interpretation; Koinzidenzlemma; Substitutionslemma für Terme (und das für Formeln); Hoare-Kalkül; Modell; (Logische) Folgerung; Allgemeingültigkeit; Folgerbarkeit
09	Pradikatenlogik: Normalformen; Negationsnormalform; Pränexe Normalform; Skolem-Normalform; Herbrand-Strukturen; Satz von Herbrand; Endlichkeitssatz der Aussagenlogik
10	Beweistheorie (Einführung)
11	Hilbertkalkül; Deduktionstheorem; Vollständigkeit der PL1; Kompaktheitssatz; Endlichkeitssatz
12	Aussagenlogik: Resolutionskalkül; 1-Resolution
13	Prädikatenlogik: Resolutionskalkül
14	Tableaukalkül
19	Prädikatenlogik: Sequenzenkalkül
21	Peano-Arithmetik; Unentscheidbarkeit
28	JML
22	Reduktionssysteme: Gleichungslogik; Satz von Birkhoff; Termersetzungssysteme; Reduktionssysteme; Kanonische Reduktionssysteme; Noethersche Induktion; (lokale) Konfluenz
23	Termersetzungssysteme; Kritische Paare
27	Modallogik; Bakery-Algorithmus; Kripke-Strukturen; Charakterisierungstheorie; Entscheidbarkeit modaler Logiken
41	(Vollständige) endliche Automaten; NEAs; Spontane Übergänge; Satz von Myhill und Büchi (vgl. Konstruktion); Reguläre Ausdrücke
42	Büchi-Automaten; Zerlegungssatz
43	Lineare Temporale Logik; omega-Struktur; LTL-Formeln; LTL-Semantik;
45	LTL und Büchi-Automaten
50	Wiederholung

Übungsblätter

Übungsblatt	Lsg	Inhalt
ÜB 1: Aussagenlogik	Lsg	Erfüllbarkeit, Unerfüllbarkeit, Allgemeingültigkeit, Tautologie, KNF, DNF, Interpolanten
ÜB 2: Aussagenlogik	Lsg	KKNF, BDD, Shannon Graphen
ÜB 3: Aussagenlogik, PL1	Lsg	DNF, KNF, DPLL
ÜB 4: PL1	Lsg	PL1 (Variante des Zebrarätsels); Unifikation
ÜB 5: PL1	Lsg	Verwandschaftsbeziehungen; (Java) Integer; Interpretation/Modell/Formel; Erfüllbar / allgemeingültig / unerfüllbar
ÜB 6: PL1	Lsg	Pränexnormalform; Skolemnormalform (09)
ÜB 7: PL1, Aussagenlogik	Lsg	Hilbertkalkül (11); Resolutionskalkül (12)
ÜB 8: PL1	Lsg	Resolutionskalkül, Tableaukalkül (13, 14)
ÜB 9: PL1, Aussagenlogik	Lsg	Resolutionskalkül, Sequenzenkalkül (19)
ÜB 10: JML	Lsg	Klassen-Invarianten; Methoden-Vertrag; Spezifikation
ÜB 11: Reduktionssysteme	Lsg	reflexive, transitive Hülle; (lokal) konfluent; noetersch; irreduzibel; Ackermann-Funktion; noethersche Induktion (22)
ÜB 12: Modallogik	Lsg	Kripke-Strukturen (27)
ÜB 13: Büchi-Automaten	Lsg	$\omega$-Sprachen, $\omega$-reguläre Ausdrücke (41, 42)
ÜB 14: LTL	Lsg	LTL-Formeln und Büchi-Automaten (43, 45)

JML

Ein paar Auszüge aus den Folien von Prof. Dr. Beckert. Ich finde daran sieht man schön, wie JML funktioniert:

public class PostInc{
    public PostInc rec;
    public int x, y;
    /*@ public invariant x >= 0 && y >= 0 &&
      @ rec.x >= 0 && rec.y >= 0;
      @*/

    /*@ public normal_behavior
      @ requires true;
      @ ensures rec.x == \old(rec.y) &&
      @ rec.y == \old(rec.y) + 1;
      @*/
    public void postinc() {
        rec.x = rec.y++;
    }
}

Quelle: 28JML.pdf#page=3

Wichtig ist noch:

/*@
  @ assignable \nothing;
  @ ensures (\forall int j; l <= j && j < \result; a1[j] != a2[j] );
  @*/

Das assignable \nothing besagt, dass keine Werte (nach außen sichtbar) verändert werden dürfen.

Die Schleifen-Syntax ist \forall int i; B; R, wobei B eine Bereichseinschränkung und R der Schleifenrumpf ist. Es gibt auch noch \exists int i; B; R.

Die Prädikatenlogischen Operatoren sind

!: Negation
&&: und
||: oder
==>: Implikation
<==>: Äquivalenz
==: Gleichheit

public int commonEntry(int l, int r) {
    int k = l;

    /*@ loop_invariant
      @ l <= k && k <= r &&
      @ (\forall int i; l<=i && i<k; a1[i] != a2[i]);
      @ assignable \nothing;
      @ decreases a1.length - k;
      @*/
    while(k < r) {
        if(a1[k] == a2[k]){break;}
        k++;
    }
    return k;
}

Quelle: 28JML.pdf#page=44

Modallogik

Interpretation	$\square A$ bzw. $\square_p A$	$\diamond A$
	A ist notwendigerweise wahr	A ist möglicherweise wahr
Zeit	A ist zu jedem zukünfigem Zeitpunkt wahr	Es gibt einen zukünftigen Zeitpunkt zu dem A wahr ist
Glauben	Ein Agent p glaubt A	A ist konsistent mit den Aussagen, die p für wahr hält
Wissen	Ein Agent p weiß A	p weiß nicht, dass A falsch ist
Programmausführung	Nach Ausführung des Programs p gilt A	Es gibt eine Ausführung des Programs p, nach der A wahr ist

Kurz und Gut

Die folgenden Stichpunkte sollte man (größtenteils nur sinngemäß) auswendig können und verstehen:

Eine Signatur $\Sigma$ ist eine abzählbare Menge von Symbolen. Die Elemente der Signatur heißen “Aussagevariablen”.
Eine Interpretation ist eine Abbildung $I: \Sigma \rightarrow {W, F}$
Ein Modell einer Formel $A \in For0_\Sigma$ ist eine Interpretation $I$ mit $val_I(A) = W$.
Ein Modell einer Formel ist also einfach eine Variablenbelegung, welche die Formel erfüllt.
Für $M \subseteq For0_\Sigma, A \in For0_\Sigma$ gilt: $M \models A$ (lies: aus M folgt A), falls jede Variablenbelegung, welche $M$ erfüllt, auch $A$ erfüllt.
$A \rightarrow B \equiv \neg A \lor B$
$A \models B$ gdw. $\models A \rightarrow B$
Der shannon-Operator $sh(a,b,c)$ ist if(a) {c} else {b}.
Die Craig-Interpolation von $A \rightarrow B$ ersetzt alle Aussagevariablen ${\text{Aussagevariable } a \in A | a \notin B}$ mit $c_i$ ($i=1,\dots,n$). Die Interpolante ist dann $C := \bigvee_{(c_1, \dots, c_n) \in {0,1}^n} A[c_1, \dots, c_n]$.
Eine DNF heißt “vollständig” bzgl. einer Signatur $\Sigma$, wenn falls für jedes $P \in \Sigma$ in jeder Klausel entweder $P$ oder $\neg P$ vorkommt.
Eine DNF heißt “minimal”, wenn jede kürzere Formel nicht äquivalent ist.
KKNF-Konstruktion: (1) Shortcuts $Q_1, \dots, Q_n$ für binäre Operatoren erstellen. Diese Shortcuts dürfen auch andere Shortcuts verwenden (2) Äquivalenzen auflösen (3) In KNF umformen.
$sh(P_i, A, B)$ heißt normiert, wenn $A$ und $B$ normiert sind und jede in $A \cup B$ vorkommende Variable $P_j$ gilt $i < j$.
Ein Shannon-Graph heißt reduziert, wenn es keine zwei Knoten $v,w$ gibt, sodass die beiden in $v$ und $w$ verwurzelten Teilbäume isomorph sind und es auch keinen Knoten gibt, bei dem beide ausgehenden Kanten in den selben Nachfolger führen.
Reduzierter Shannon-Graph = OBDD = BDD = ordered binary decisio diagram
Bei gegebener Indizierung sind reduzierte Shannon-Graphen bis auf Isomorphie eindeutig. Ist die Indizierung nicht gegeben, macht die Variablenanordnung einen großen Unterschied in der Größe (Knotenmenge) des reduzierten Shannon-Graphen.
Multiplikation $k$-stelliger Binärzahlen: Für jede Ordnung $<$ der Variablen in $X={x_0, \dots, x_{k-1}, y_0, \dots, y_{k-1}}$ gibt es einen Index $0 \leq i < 2k$, sodass der BDD $B_{Mult_i,<}$ mindestens $2^{k/8}$ Knoten besitzt.
Horn-Formel: Formel in KNF, wobei jede Klausel höchstens ein positives Literal enthält.
Negationsnormalform: Negationen nur vor Atomen.
Bereinigte Formel: (1) $Frei(A) \cap Bd(A) = \emptyset$ (2) Die hinter Quantoren stehenden Variablen sind paarweise verschieden.
Pränexe Normalform: $A = Q_1 x_1 Q_2 x_2 Q_3 x_3 \dots Q_n x_n B$, wobei $B$ quantorenfrei sein muss. Dann heißt $B$ die Matrix von $A$.
Die Pränexe Normalform ist nicht eindeutig.
Skolem-Normalform: (1) geschlossene Formel (2) $\forall x_1 \dots \forall x_n B$ (3) Matrix $B$ ist in KNF
Gödelscher Vollständigkeitssatz: Es gibt einen Kalkül der PL1 derart, dass für jede Formelmenge $\Gamma$ und für jede Formel $\varphi$ gilt: $\varphi$ folgt genau dann aus $\Gamma$, wenn $\varphi$ im Kalkül aus $\Gamma$ hergeleitet werden kann. In Zeichen: $\Gamma \models \varphi \Leftrightarrow \Gamma \vdash \varphi$.
Gödelscher Unvollständigkeitssatz:
- Jedes hinreichend mächtige, rekursiv aufzählbare formale System ist entweder widersprüchlich oder unvollständig.
- Jedes hinreichend mächtige konsistente formale System kann die eigene Konsistenz nicht beweisen.
Kompaktheitssatz: Wenn A aus einer unendlichen Teilmenge der Formelmenge folgt, dann auch aus einer endlichen.
Endlichkeitssatz: Eine Menge $M \subseteq For_\Sigma$ hat genau dann ein Modell, wenn jede endliche Teilmenge von $M$ ein Modell hat.
Der Resolutionskalkül arbeitet nur mit Formeln in Skolemnormalform.
Tableau-Kalkül:
- Typ-$\alpha$: Alles, was keine Quantoren hat und eindeutig ist
- Typ-$\beta$: Alles, was keine Quantoren hat, aber nicht eindeutig ist
- Typ-$\gamma$: Unendlich viele
- Typ-$\delta$: min. eines
Ob eine prädikatenlogische Formel allgemeingültig ist, ist unentscheidbar.
Die Menge der allgemeingültigen prädikatenlogischen Formeln ist rekursiv aufzählbar.
Die Menge der erfüllbaren prädikatenlogischen Formeln ist nicht rekursiv aufzählbar.
Ein Reduktionssystem ist ein Tupel $(D, \succ)$, wobei $D \neq \emptyset$ eine Menge ist und $\succ$ eine Relation auf $D$ ist.
$\rightarrow$ bezeichnet die reflexive, transitive Hülle von $\succ$.
$\stackrel{+}{\rightarrow}$ bezeichnet die transitive Hülle von $\succ$.
$\leftrightarrow$ bezeichnet die reflexive, transtive und symmetrische Hülle von $\succ$.
$(D, \succ)$ heißt konfluent $:\Leftrightarrow \forall s_1, s_2, s_3 \in D$ mit $s \rightarrow s_1 \land s \rightarrow s_2 \exists t \in D: s_1 \rightarrow t \land s_2 \rightarrow t$
$(D, \succ)$ heißt lokal konfluent $:\Leftrightarrow \forall s_1, s_2, s_3 \in D$ mit $s \succ s_1 \land s \succ s_2 \exists t \in D: s_1 \rightarrow t \land s_2 \rightarrow t$
$(D, \succ)$ heißt noethersch, wenn es keine unendlichen Folgen $s_0 \succ s_1 \dots \succ s_i \succ \dots$ gibt.
Ein konfluentes und noethersches Reduktionssystem heißt kanonisch.
Ein Element $s \in D$ heißt irreduzibel (oder eine Normalform) in $(D, \succ)$, wenn kein $t \in D$ existiert mit $s \succ t$.
Sei $s \in D$. Ein Element $s_0 \in D$ heißt eine Normalform für $s$ in $(D, \succ)$, wenn $s_0$ irreduzibel ist und $s \rightarrow s_0$ gilt.
In kanonischen Reduktionssystemen hat jedes Element eine eindeutige Normalform.
$(D, \succ)$ ist noethersch und lokal konfluent $\Rightarrow (D, \succ)$ ist konfluent.
Eine Kripke-Struktur ist ein Tupel $\mathscr{K} = (S, R, I)$ mit
- $S \neq \emptyset$ ist die Menge der Zustände bzw. der möglichen Welten. $s \in S$ heißt also auch eine “Welt”.
- $R \subseteq S \times S$ ist die Zugänglichkeitsrelation
- $I: (\Sigma \times S) \rightarrow {W, F}$ ist die Interpretation der Aussagenlogischen Variablen
$(S, R)$ heißt der Kripke-Rahmen von $\mathscr{K}$.
$\square A \rightarrow A$ ist nur in reflexiven Kripke-Strukturen eine Tautologie.
Ein endlicher Automat ist ein Tupel $(S, V, \delta, s_0, S_1)$, wobei
- $S$ eine endliche Zustandsmenge ist,
- $V$ ein endliches Alphabet (terminale Zeichen) ist,
- $\delta: S \times V \rightarrow S$ eine Funktion ist, die besagt bei welchem Eingabezeichen man von welchem Zustand aus in welchen Zustand kommt,
- $s_0 \in S$ ein Startzustand und
- $S_1 \subseteq S$ die Menge der Endzustände ist
$V^\omega$ ist die Menge der unendlichen Wörter mit Buchstaben aus $V$.
$w(n)$ ist der $n$-te Buchstabe des Wortes $w$.
$w \downarrow (n)$ ist das endliche Anfangsstück $w(0)\dots w(n)$ von $w$.
$\varepsilon \notin V^\omega$
Für $K \in V^*$ und $J \in V^\omega$ ist $KJ={w_1 w_2 w_1 \in K, w_2 \in J}$
Für $K \in V^*$ ist $\overset{\rightarrow}{K}={w \in V^\omega w \downarrow (n) \in K \text{ für unendlich viele } n}$
Ein Büchi-Automat ist ein nicht deterministischer endlicher Automat, der Wörter akzeptiert, wenn es eine Berechnungsfolge mit unendlich vielen Finalzuständen gibt. (vgl. Beispiel)
A U B: A gilt, bis B gilt (das U steht für “until”). Allerdings gilt B auch irgendwann. Es kann also nicht sein, dass A unendlich lange gilt und B nie.
$A\;\textbf{U}_W\;B$ bedeutet, dass entweder immer A und gleichzeitig nie B gilt, oder dass irgendwann B gilt und davor gilt immer A. Das ‘W’ steht für ‘weak’.
$A\;\textbf{V}\;B$: B gilt so lange, bis A gilt. Daher wird V auch ‘Release-Operator’ genannt.
$\diamond \square P$: Es gibt einen Zeitpunkt, ab dem immer B gilt.
Zu jeder LTL-Formel gibt es einen effektv konstruierbaren Büchi-Automaten.
Erfüllbarkeit und Allgmeingültigkeit von LTL-Formeln ist entscheidbar.
Ein Kalkül ist korrekt, wenn alles was formal ableitbar auch wahr ist. (vgl. Korrektheit (Logik))
08Pk1Semantik-print.pdf, Folie 34/37: QxB steht für “Quantor x B”, wobei der Quantor entweder $\exists$ oder $\forall$ ist, $x$ eine Variable ist und $B$ eine Formel ist.

Wissens-Check

Folgende Fragen sollte man für die Klausur schnell beantworten können:

Nenne 3 Basen für die Aussagenlogik. Eine davon soll höchstens einen Operator haben.
In welcher Komplexitätsklasse ist das Erfüllbarkeitsproblem für 3-KNF? In welcher 2-KNF? Wie sieht es mit dem Allgemeingültigkeitsproblemen aus?
Was ist der Shannon-Graph von $1$?
Was ist der Shannon-Graph von $0$?
Was ist der Shannon-Graph von $a \lor b$?
Was ist der Shannon-Graph von $a \land b$?

Meine Fragen

02, Folie 19/29: Warum wird einmal $\models$ und dann $\models_\Sigma$ geschrieben?
07, Folie 19/51: Warum “fast alle” $x \in Var$? Was bedeutet das?
- Es dürfen nur endlich viele Variablen umbenannt bzw. durch einen Term ersetzt werden. Andernfalls werden die Beweise und Notationen hässlich. Abzählbar unendlich viele Variablen sind praktisch, wenn man sich die Existenz immer weiterer Variablen sichern will, die nicht in einer gegebenen Formel auftauchen. Natürlich ist jede Formel endlich, enthält also nur endlich viele Variablen.
Folie 8/30: Wieso stimmt diese Umformung?
- Gebundene Variablen dürfen unabhängig von freien umbenannt werden. Die andere Umformung kann nachvollzogen werden, wenn die Implikation $A \rightarrow B$ zu $\neg A \vee B$ umgeformt wird. Man beachte folgende Umformungsregel: $\neg \forall x F(x) \equiv \exists x \neg F(x)$.
Folie 23/30: Was ist eine Grundinstanz? Wo ist der Unterschied zwischen “Grundinstanz” und “Instanz”? Was sind “Grundterme”?
- Grundterm: Ein Term, der keine Variablen enthält. Instanz: Für quantifizierte Variablen wurden Terme eingesetzt. Grundinstanz: Für alle Variablen wurden Grundterme eingesetzt. Damit enthalten Grundinstanzen überhaupt keine Variablen mehr.
Folie 24/30: Was ist ein Beispiel für $D = Term_\Sigma^0 \neq$ Menge der Grundterme? Wo gilt 2. nicht?
- Ich vermute mal, dass $Term_\Sigma^0 := $ Menge der Grundterme. Bachte (sofern die Definition stimmt): $Term_\Sigma^0 \subseteq Term_\Sigma$, da es auch Terme gibt, die Variablen enthalten, falls welche in der Signatur vorhanden sind. Was ist die Bedeutung von Herbrand-Strukturen / dem Satz von Herbrand?
Blatt 6, Lösung zu Aufgabe 4: Den Teil mit der Umwandlung einer Aussagenlogischen Formel verstehe ich nicht. Kann das jemand bitte für $a \land \neg b \lor c \lor d$ erklären?
Blatt 9, Lösung zu Aufgabe 1: Ist der Baum, also insbesondere die ersten 4 Knoten, richtig? Warum steht in Knoten 1 nicht $1\forall x \forall y \forall z (r(x,y) \land r(y,z) \rightarrow r(x,z))$? Wie funktioniert der 1. Schritt in Aufgabe 2?
Haben reflexive Relationen irreduzible Elemente?
- Laut unserer Definition nicht. Aber du kannst jede n-stellige Relation mit dem Komplement der n-stelligen Gleichheits-Relation schneiden, dann erhältst du ihr irreflexives Gegenstück.
Können reflexive Relationen noethersch sein?
- Nur, wenn sie leer sind. Ansonsten gibt es immer unendliche Ketten.
Folie 27: Wie muss ich $\square \diamond P$ lesen?
- Von jeder erreichbaren Welt aus gibt es eine erreichbare Welt, in der P gilt. Die Verbalisierung der Relation “erreichbar” klappt immer ;)
Blatt 12, Aufgabe 1b: Was sagt $\diamond\square P$ auf dem Graphen aus? Insbesondere: Warum ist $w_5$ nicht in $\diamond\square P$? Was wäre $[[\square \diamond P]]$?
Blatt 12, Aufgabe 4: Das muss ich noch mal in Ruhe durchgehen.
21, Folie 8: Was ist $Th(N)$ und was ist $Cn(PA)$?
- $Th(N)$: Theoreme über N, also die Menge aller Formeln, für die die natürlichen Zahlen ein Modell sind. Cn(PA): Menge aller Formeln, die aus den Axiomen der Peano-Artihmetik gefolgert werden können. Da die Peano-Arithmetik korrekt ist, ist jede Formel aus Cn(PA) auch in Th(N).
50, Folie 16: Was bedeutet es, dass $Th(N)$ nicht rekursiv ist?
- Rekursiv heißt entscheidbar. Da die Peano-Axiome durch die Peano-Artihmetik formalisiert werden können, gibt es eine Formel P(x), die genau dann wahr ist, wenn für x die Kodierung einer solchen Formel eingesetzt wird, die sich nicht aus den Peano-Axiomen herleiten lässt. Nach dem Gödelschen Unvollständigkeitssatz gibt es nun eine Formel P und x derart, dass x mit der Kodierung von P(x) übereinstimmt: Eine Formel also, die über sich selbst behauptet, sie sei nicht herleitbar. Da es nur ein Modell gibt, ist jede Formel entweder unerfüllbar oder allgemeingültig. Wenn $Th(N)$ entscheidbar wäre und P(x) allgemeingültig, dann entsteht Widerspruch zur Wahl von P und x. Wenn P(x) unerfüllbar wäre, ist nach Wahl von P und x P(x) herleitbar, was wieder einen Widerspruch darstellt. Um den Widerspruch aufzulösen, darf “aus den PA-Axiomen herleitbar” nicht mit “im Modell der nat. Zahlen gültig” übersetzt werden.
23, Folie 5: Kann mir jemand ein konkretes Beispiel geben?
43, Folie 2: Was sind omega-Strutkuren und insbesondere was bedeutet $2^P$?
- Eine Omega Struktur Ordnet jedem Zeitpunkt, wenn die natürlichen Zahlen als Zeitstrahl aufgefasst werden, eine Menge von aussagenlogischen Variablen zu, die als “wahr” gelten sollen. $2^P$ ist die Potenzmenge von P.
Ist $A\;\textbf{U}_W\;B$ äquivalent zu $B\;\textbf{V}\;A$?
Blatt 14, 2a: Warum ist $\diamond (p\;\textbf{U}\;q)$ äquivalent zu $\diamond q$? Ich dachte es wäre äquivalent zu $p\;\textbf{U}\;q$?
Blatt 14, 3: Wie würde der Automat aussehen, wenn das $X$ weggelassen würde?
Blatt 14, 4: Das würde ich gerne gemeinsam durchgehen.
50, 13: Was sind die Ziele der Beweistheorie? Was ist die Grundidee des Hilbert-Kalküls?
- Ziele: Automatisches Beweisen ermöglichen und die Grenzen bestimmen. Grundidee des Hilbert-Kalküls: Aus Axiomen konstruktiv Formeln folgern. Im Gegensatz zu den praxis-orientierten Beweisverfahren kann der Hilbert-Kalkül leichter in theoretischen Beweisen verwendet werden. Der Gödelsche Unvollständigkeitssatz verwendet z.B. den Hilbert-Kalkül.
50, 13: “Aussagenlogische Tableauregeln aus Wahrheitstafeln konstruieren.” - was ist damit gemeint?
50, 15: Was ist die Grundidee der Peano-Arithmetik?
- Liegt doch eigentlich auf der Hand: Zahlentheoretische Aussagen automatisch beweisen.
50, 33: Wie kann man sich in dem Beispiel den Unterschied zwischen Prädikaten und Funktionen erschließen?
- Funktionen sind immer in Prädikaten enthalten. Ein Prädikat darf kein weiteres Prädikat enthalten.
Wie ist Erfüllbarkeit in der Prädikatenlogik definiert? Z.B. erscheint mir $\exists x p(x)$ erfüllbar, wenn nichts weiter gegeben ist. Wenn man aber $p(x)=False$ für alle $x$ sagt, dann ist es unerfüllbar.
- F ist erfüllbar gdw. es ein Modell für F gibt, d.h. eine Interpretation I, sodass val_I(F) = W.
Müssen die Kanten von Shannon-Graphen mit W und F oder mit 1 und 0 beschriftet werden? Praktisch alle Altklausuren verwenden 1 und 0, aber ich meine er hätte in der Vorlesung gesagt, dass wir W und F verwenden sollen.
Ist die Menge der allgemeingültigen / erfüllbaren / unerfüllbaren Formeln der PL1 abzählbar? Jede Teilmenge aus $\Sigma^*$ ist abzählbar für ein endliches Alphabet $\Sigma$. Es gibt aber überabzählbar viele Teilmengen, deren Elemente zwar wieder abzählbar sind, aber sowohl die Teilmengen selbst als auch ihre Elemente nicht aufzählbar sind. Beachte: Aufzählbar $\neq$ Abzählbar.
Wie kann man $U$ durch $U_W$ darstellen?

Altklausuren

WS2010/2011, 1. Zwischentest, A4: Ist der Markierungsalgorithmus für Hornformeln relevant?
WS2010/2011, 2. Zwischentest, A1:
- Was ist eine kompakte Logik? (PL2 ist laut WS2008/2009, 2.Zwischentest 1b, nicht kompakt)
- Was bedeutet $Cl_{\exists}(t_1 \doteq t_2)$?
WS2009/2010, A1c: Wieso gibt es für $\exists p(x)$ kein Herbrandmodell?
WS2009/2010, 1.Zwischentest
- A1a: Was ist ein “Vereinfachungsschritt” im Davis-Putnam-Verfahren? Was bedeutet es, wenn dort keine Klausel mehr zur Verfügung steht?
- A1a: “Wenn A und B erfüllbar sind, dann ist auch $A \rightarrow B$ erfüllbar.” - Was ist mit $B = \neg A$?
WS 2009/2010, 2. Zwischentest (ist beim ersten):
- A1a: Warum ist $\forall x \forall y (x \leftrightarrow y)$ keine Formel der PL1?
- A4: Will ich gerne durchsprechen
WS 2008/2009:
- 1a: Warum ist $\forall x (p(x) \rightarrow q(p(x)))$ keine Formel der PL1?
- 1a: Warum ist $\exists x (p(x) \rightarrow p(f(x)))$ allgemeingültig?
- 1b: Was ist ein Beispiel für einen Büchi-Automaten, für den es keinen deterministischen Büchi-Automaten gibt?
- 1b: “Für jede geschlossene prädikatenlogische Formel G gilt: Es gibt ein Modell für G oder für das Negat von G oder für beide.” - Gilt das nicht für alle prädikatenlogischen Formeln (egal ob geschlossen oder nicht)?
WS 2008/2009, 1. Zwischentest:
- 1a: Wie lange dauert der kürzeste Resoultionsbeweis in Anzahl der Literale?
- 1b: Warum gibt es keine prädikatenlogische Interprätation, in der alle prädikatenlogischen Formeln wahr sind?

Material

StackExchange:

Übungsbetrieb

Wo sind die Übungsblätter: Link
Abgabeform: Keine Abgabe
Turnus: wöchentlich
Lösungen: Die Lösungen von jeweils zwei Blättern werden dann in den 14-tägig stattfinden Übungen am Freitag besprochen.
Übungsschein verpflichtend: Es gibt keinen Übungsschein.
Bonus durch Übungsschein: Es gibt keinen Klausurbonus durch Übungsblätter.
Anderer Klausurbonus: Man kann durch insgesammt 4 Zwischentests und 2 Praxisaufgaben für die wirkliche Klausur Punkte sammeln. Die Teilnahme an den Zwischentests und den Praxisaufgaben ist freiwillig. Die erzielten Übungspunkte werden im Verhältnis 1:10 als Bonuspunkte auf die bestandene Abschlussklausur angerechnet.

Termine und Klausurablauf

Siehe Klausurtermine-Seite für zukünftige Termine.

Datum: Freitag, den 6. März 2015 von 11:00 bis 12:00 Uhr (Quelle).
Ort: Gerthsen (30.21) und HSaF (50.35) - vgl. Anmeldeliste
Punkte: 60
Punkteverteilung: ? (Stand: 06.03.2015)
Bestehensgrenze: ? (Stand: 06.03.2015)
Übungsschein: Gibt es nicht.
Bonuspunkte: Bis zu 8. Die Punkte aus den 4 Zwischenprüfungen und 2 Praxisaufgaben werden addiert, dann durch 10 geteilt und schließlich kaufmännisch gerundet.
Ergebnisse: Klausur wurde noch nicht geschrieben
Einsicht: steht noch nicht fest (Stand: 06.03.2015)
Erlaubte Hilfsmittel: Keine.

Ergebnisse

Klausur wurde noch nicht geschrieben.

Gradient Descent, the Delta Rule and Backpropagation

info@martin-thoma.de (Martin Thoma) — Sun, 26 Oct 2014 21:06:00 +0100

If you learn about machine learning you will stumble over three terms that are related:

Gradient descent,
the Delta rule and
backpropagation

Gradient descent is a way to find a minimum in a high-dimensional space. You go in direction of the steepest descent.

The delta rule is an update rule for single layer perceptrons. It makes use of gradient descent.

Backpropagation is a special form of gradient descent, where a rule can be formulated which has some recursively defined parts. Those parts belong to neurons of different layers and get calculated from the output-layer (last layer) to the first hidden layer.

On-line Handwriting Recognition of Mathematical Symbols

info@martin-thoma.de (Martin Thoma) — Fri, 17 Oct 2014 12:25:00 +0200

On-line handwriting recognition systems get the information how a symbol is written. In contrast, OCR only gets the pixel map.

I’ve created a system that can be used to work with handwriting recognition systems in my bachelor’s thesis.

write-math.com

The website write-math.com was used to collect data. The source is at github.com/MartinThoma/write-math.

hwrt toolkit

The hwrt toolkit was created to work with on-line handwritten symbols. The toolkit is documented at pythonhosted.org/hwrt.

The raw data can be downloaded with this toolkit.

The toolkit can be used to classify data on your computer (without internet connection):

Write math: Interactive Browser Interface (offline)

nntoolkit

The nntoolkit was created to have a free software to create, train, test and evaluate neural networks.

HWR experiments

All experiments configuration files are saved in the project github.com/MartinThoma/hwr-experiments.

Data

The data can be downloaded from write-math.com/data. I will try to keep a relatively recent version online. You can contact me if you want the latest version. However, I should note that currently (2015-04-12) this is about 3.7GB. This means sharing the data is not that easy.

Presentations

27.08.2014
06.11.2014: Final presentation for bachelor’s thesis

Bachelor’s thesis

07.11.2014: My bachelor’s thesis. I’ve got the best grade (1.0) for it ☺. Please note that the submission to arxiv was later and a couple of typos were fixed as well as the term “data multiplication” was replaced by “data augmentation”.
29.06.2015: An updated, condensed version of my bachelor’s thesis.

Remarks

What I called “data multiplication” is called “data augmentation” by others (e.g. ImageNet Classification with Deep Convolutional Neural Networks, Deep Image: Scaling up Image Recognition, Classifying plankton with deep neural networks)

3D Printing

info@martin-thoma.de (Martin Thoma) — Sun, 28 Sep 2014 22:23:00 +0200

3D printing is a hot topic for quite a while now. 3D printers have become cheap enough so that many ordinary people can have 3D printers for their personal use. In this article, I would like to point out some very nice applications of 3D printing.

Medicine

3D printing has basically two application in medicine: Very customized non-organic parts or organic printing.

Printing a human Kidney

“Magic Arms” prosthetic

3D printing gives man new face

Other examples

Hearing aids
Bones
Print on wounds

Shoes

I am always struggling with finding good shoes. Most shoes I try feel unconfortable, although I have size 42 (which seems to be a very standard size in Germany for males). What if we could go to a store, let them 3D scan our feet and they automatically adjust a 3D model of a shoe to fit to our feet. Then some parts are 3D printed, assembled and a few days later we get a shoe that is customized to our feet and cheap.

continuumfashion.com seems to 3D print shoes with Nylon. kobilevidesign.com is another designer who 3D prints shoes.

Toys

Board game figures (e.g. custom chess sets, Monopoly cars, …)
Model building (e.g. Ship model where people pay A LOT of money for it)
Lego

Teaching

Did you have chemistry in school and learn about atom models / crystaline structures? With 3D printing, it becomes easy to get those models for every student!

Or think about architecture and history. You could show models of ancient cities or buildings in 3D.

Spare parts

Imagine something small breaks. Sometimes very expensive devices that are otherwise fine, become useless or at least ugly. Now think you could go online, download a 3D model of that part and just print it.

Design

Lamps
Cups

Weapons

Sadly, 3D printing can also be used to produce weapons (source).

However, a 3D printed pistol has a high chance of not working at all, of hurting the person that created it and of working. So currently, it’s quite a lot of gambling. But that might change when the quality of 3D printing improves.

interACT

info@martin-thoma.de (Martin Thoma) — Thu, 11 Sep 2014 17:25:00 +0200

interACT is an exchange program for students. I’m a student from KIT (Germany) and I went to CMU in Pittsburgh (US). The information in the following article will be most useful for students who also come from KIT and want to go to CMU.

Participating universities

An interACT exchange is available between the following universities:

United States:
- CMU, Pittsburgh and Mountain View
- USC (Los Angeles)
Germany: KIT (Karlsruhe)
China: HKUST (Hong Kong)
Japan
- Waseda University (Tokyo)
- NICT (Tokyo)
- NAIST (Ikoma)
Spain: IIT (Madrid)

Preparation

You need to think about quite a lot for your preparation. This should give you an overview:

Forms and other general stuff

bank statement
send copy of passport to interACT
visit US embassy
get to know some people in the US before you get there
transfer money on your VISA card to make sure you have enough when you’re there
mobile: ultra.me is the best internet / phone provider!
test internet calls (Skype / Google Hangouts) with parents and partner
organize the flight after you have the Visa back

Find a room

Do you want to find a subleaser?
Get a room in the US
- http://sfbay.craigslist.org/
- eventually airbnb.de for the first nights

Health

Get health insurance for the US
- Signal Iduna
- barmenia.de:
  - http://www.barmenia.de/de/produkte/reiseversicherung/einzelreisen/reisekrankenversicherung.xhtml
  - price: ca. 283 Euro
- If you are also in the “Studienstiftung” you should have a look in “Daidalosnet”. There is a very good offer for health insurance
Go to all doctors you need in Germany right before you go to the US:
- dentist
- Vaccination (I refreshed Tetanus, Diphtheria and Pertussis with a single injection)

What to take with you

Cloths

Check the weather and adjust the list to your needs. Don’t forget

a raincoat
warm cloths for the flight

And, of course, don’t forget towels:

Just about the most massively useful thing any interstellar Hitchhiker can carry. Partly it has great practical value. You can wrap it around you for warmth as you bound across the cold moons of Jaglan Beta; you can lie on it on the brilliant marble-sanded beaches of Santraginus V, inhaling the heady sea vapours; you can sleep under it beneath the stars which shine so redly on the desert world of Kakrafoon; use it to sail a miniraft down the slow heavy River Moth; wet it for use in hand-to-hand combat; wrap it round your head to ward off noxious fumes or avoid the gaze of the Ravenous Bugblatter Beast of Traal (a mind-bogglingly stupid animal, it assumes that if you can’t see it, it can’t see you — daft as a brush, but very very ravenous); you can wave your towel in emergencies as a distress signal, and of course you can dry yourself off with it if it still seems to be clean enough.

More importantly, a towel has immense psychological value. For some reason, if a strag (strag: nonhitchhiker) discovers that a hitchhiker has his towel with him, he will automatically assume that he is also in possession of a toothbrush, washcloth, soap, tin of biscuits, flask, compass, map, ball of string, gnat spray, wet-weather gear, space suit etc., etc. Furthermore, the strag will then happily lend the hitchhiker any of these or a dozen other items that the hitchhiker might accidentally have “lost.” What the strag will think is that any man who can hitch the length and breadth of the Galaxy, rough it, slum it, struggle against terrible odds, win through and still knows where his towel is, is clearly a man to be reckoned with.

Hence a phrase which has passed into hitch hiking slang, as in “Hey, you sass that hoopy Ford Prefect? There’s a frood who really knows where his towel is.”

passport
document for your health insurance in the US
smartphone
camera - I bought the Panasonic Lumix DMC-TZ41 for 260 Euro and 64 GB Flash storage for 32 Euro for this trip. It is worth the money! It is an exceptional camera!
laptop
chargers
power adapters (at least 2!). I can recommend this big adapter. Don’t take the small ones! They are very loose.
hygiene products (toothbrush, dental floss, dental floss, razor, shaving foam)

Overview by time

On 10.01.2014 I decided that I would like to go to the US. At 26.04.2014 I got my passport with the visa back. So it took more than 4 months until I could finally be sure that I can go to the US. Keep that in mind!

I have written down when I did what (that sounds soo wrong - please correct me if you know how to express that in English). It should help you to plan your trip.

Application

A first step is the application as described here. A general contact person is Margit Rödder.

Get all required documents to OIE

After you’ve got accepted by the interACT advisory board (whoever that is), you have to get a ‘bank statement’. These are some lines where your bank confirms that you have enough money to live from when you’re in the US. It is very important that this statement is in English. So you should have about 1775 US-Dollar per month on your bank account. This means, if you want to be in the US for 4 months, make sure that you have at least 7100 US-Dollar on your account

For customers of ‘Sparkasse’ this costs 25 Euro and you have to make sure that what you get is in English.
For customers of ‘comdirect’ this costs 10 Euro
For customers of ‘Volksbank / Raiffeisenbank’ this seems to be free.

So I gave the bank statement and a copy of my passport to Margit Rödder.

Some days later I’ve received an invitation from my advisor at CMU and some legal stuff I had to sign. I did this and send a copy to Faith Bold.

26.03.2014 All documents are signed and I sent them to Faith.

26.03.2014 Faith submitted the documents to the OIE.

Receive documents from OIE

03.04.2014: Got e-mail from UPS that they got my forms. Nice to know.

07.04.2014: I’ve got my forms now (especially DS-2019). It seems as if I have to pay a fee of 180 US-Dollar for SEVIS. There is no refunding possible when I don’t get the visa (source). And that’s not the same as the visa application fee (which is additional 160.00 US-Dollar).

So I’ve paid the SEVIS fee on www.FMJfee.com. You need one of the following credit cards for that step:

American Express
Diners Club
Discover
MasterCard
Visa

DS-160

14.04.2014: I’ve filled this form.

Address: You can fill in the address of your department (see admission document header)
Know your rights video and information

Visa and Visiting the U.S. embassy

You have to pay here for your visa. Some hints:

Your exchange number usually starts with a P
SEVIS number is on top of DS-2019 (N….)

Some first information: link

There is quite a lot of stuff you may not bring to the embassy (list)

You can see visa wait times online. I had to go to Frankfurt and that were about 2 days.

You do not need a DS-7002, because that’s only for interns. You’re a short-term scholar.

After-VISA-Stuff

26.04.2014

My passport arrived with the visa in it. Now I’ve contacted about 20 people I found on craigslist asking for a room in Pittsburgh.

27.04.2014

Finding a room to live fro 3 months in Pittsburgh is more difficult than I thought.

And finding a room to store my stuff for those 3 months in Karlsruhe is also not so easy. All self-storage services like lager7, lagerbox.com and deinlagerplatz.de are not in Karlsruhe. But I thought finding a cellar / garage near Karlsruhe for 50 Euro / month or less would be easy …

edit: I finally decided to store most of my stuff at my fathers and sublease the room. That was a good decision. Here are some documents for subleasing. But before you start ask your landlord if you’re allowed to sublease!

07.05.2014

An e-mail arrived from “Baden-Württemberg-Stipendium” telling me that I now have full access to their material and that I have to sign something and sent it back to them.

They also told me that my scholarship is 900 Euro / month for 4 months. Finally some good news! Eventually it will not get as expensive as I thought!

Getting the flight tickets

I have heard that fluege.de has a very bad service. They seem not to send you the tickets in time (I’ve heard something about two weeks later!) and they don’t show the correct price. There is more information on Wikipedia.de.

OneTwoTrip on the other hand had some good reviews 1 2.

Here is a (German) explanation of the booking process:

Other online portals are:

skyscanner.de has only one rating on ciao.de
swoodoo.com is a meta search engine that uses fluege.de
billigflieger.de has only 3 ratings on ciao.de and 60 on trustpilot.de
expedia.de has 3/5 stars on ciao.de and 1/5 stars on trustpilot.de
fluege.de has 2.5/5 stars on ciao.de and 3/5 stars on trustpilot.de
cheaptickets.de has very bad critics on Wikipedia (see ciao.de) but 4/5 stars on trustpilot.de

Baggage: united.com

IMPORTANT If possible, try not to fly over Chicago! That Airport is HUGE. I had 1.5 hours to get my next flight and I did not manage it. It took me quite a while to realize that you have to get into a train to get to the other terminal. And you have to re-checkin your luggage. That’s annoying! (I did not have to do that when I left the US)

Preparing and flight

28.05.2014: I’ve transferred the money for the flat. All of it.

30.05.2014: I’ve went to the German airport in Munich about 2 hours earlier to make sure that I don’t miss my flight. We arrived at the correct terminal. First of all, I had to “give up” (is it called “check-in”?) my luggage. After that, I spend some time with my father.

Then, 10 minutes before the “Board time” began, I went through the security checkpoint. I had to put my backpack into a plastic box, my large electronic devices into separate ones (because the metal / electronic blocks the rays) and my belt in another one. Make sure you don’t have anything in your pockets.

After that, you can go through a metal detector. You get checked with a hand held device after that.

Next, you have to find your “gate”. This is the place where you go onto the plane.

As soon as I arrived at the correct airport I had to find my luggage. You have to know that there are signs e.g. “Luggage A-M”. Those letters “A-M” refer to the airline (in my case United), not to your name!

The next step was to get from airport to the house. As I arrived after 12p.m. I had to go by taxi which cost. Remember: You should give a tip of at least 10%, but rather 15%.

Getting into the house was the next step to take. I was really lucky that a neighbor was on the terrace and gave me her phone to call my friend who lives in that house.

First impressions

01.06.2014: We visited a some stores and Dave and Buster’s which is a video arcade.

02.06.2014: I’ve got to know Jae Cho who now has the job that Faith had before. She introduced me to everybody. After that, I was sent to Jill Lentz. She is in GHC which is short for “Gates Hillman Center”. “Gates” and “Hillman” are two buildings that are connected. Jill gave me a form from “TheHUB” which is a student service center.

I’ve just ordered a sim card at ultra.me

Trip to New York

11.06.2014: 50.15 Euro for the bus tickets and 94.42 Euro for the hotel.

We (Emanuel, David, Damir and me) went to New York by bus. That took quite a while, but it was quite cheap and the bus was comfortable. Not like some of the remote buses in Germany where you don’t have any place for yourself.

The Royal Park Hotel is very central and very cheap, but I would not recommend it. The smell was disgusting (probably from bug spray?), we had a power blackout once (eventually because 4 people wanted to charge their phones, a laptop, two tables and a camera simultaneously in one room?) and the shower was shared with other rooms.

I recommend to visit the central park. It is great!

The American Museum of Natural History is ok. You can see something about science and astronomy in the basement and around the Hayden Sphere, but not much. The rest is about stones, people, plants and animals in America. I don’t think that mammals are very interesting, so that was pretty boring for me.

It has some “Special Exhibitions” for which you have to pay in advance if you want to be able to see them. The “Dark Universe” special exhibition is probably not worth the money if you have been to a planetarium before. I have seen very similar shows in Augsburg and the movie did not contain information that was new to me.

Smoky Quartz
Wulfenite
Hayden sphere

Labradorite
Muscovite crystal structure showing cleavage
Rosenhof: Green frog dissection

There are more images from my trip to New York City on Wikipedia Commons.

Brooklyn Bridge is also interesting. You can take some nice pictures there at night:

Ground Zero is also a must-see. It is very impressive. It is the biggest human build waterfall:

Ground Zero

And - of course - liberty island and the statue of liberty. You will also get to Ellis island and the immigrant museum. That was interesting!

Money - How much does it cost?

You have to pay quite a lot in advance. I’m my case, it were 3218.06 Euro:

bank statement: 25 Euro
SEVIS fee: 180 US-Dollar = 131.45 Euro + 1.97 Euro “ENTGELD AUSLAND”
VISA application fee: 160.00 US-Dollar = 116.70 Euro
2 photos for J1 visa: 15.00 Euro
traveling to Frankfurt for Visa application: 2.30 Euro + 6.00 Euro + 8.00 Euro = 16.30 Euro
Stay one day in Frankfurt: 59.00 Euro
passport: 37.50 Euro
health insurance: 102.72 Euro
casualty insurance: 13.44 Euro
Flight tickets: 961.62 Euro
luggage scale “Soehnle 66172”: 16.99 Euro
power adapter USA2Schuko: 4.49 Euro
rent for 3 months: 1715.88 Euro

I arrived in the US with 2000 US-Dollar in traveler checks and 200 US-Dollar in cash.

I payed 1451.68 Euro for the traveler checks and 151.63 Euro for the cash. My VISA-card expenses were 338.31 Euro. I had to transfer 144.57 Euro to friends who payed the trip to New York in advance. So an upper bound for my expenses for the trip to the US is

(3218.06+1451.68+151.63+338.31+144.57) Euro = 5304.25 Euro

I got $4 \cdot 900 = 3600$ Euro from interACT. So in total I had to pay 1704.25 Euro by myself.

I got back with 950 US-Dollar in traveler checks and about 50 US-Dollar in cash. So I could get about 772 Euro back.

In the US, I kept track of some expenses. It might give you an impression how expensive things are. Here you are:

taxi from airport: 60 US-Dollar = 44.10 Euro
Food
- 3x Stack’d: 3x 10 US-Dollar = 30 US-Dollar
- Hofbräuhaus: 23 US-Dollar (for apple juice and Schweineschnitzel!)
- Crepes: 8.03 US-Dollar (for crepes with salad and water)
- Pizza: 10.5 US-Dollar (for Pizza+Cola, including 1.5 US-Dollar tip)
- UNO Pizza: 16.67 US-Dollar
- The Original Hot Dog Shop: 7.27 US-Dollar
New York
- Bus tickets: 50.15 Euro (a friend payed it)
- Subway: 30 US-Dollar
- Eating:
  - Don Giovanni: 16.40+ 3 tip = 19.40 US-Dollar
  - Texas Rotisserie & Grill:
    - Gyro Sandwich: 7.61 * 2 = 15.22 US-Dollar
    - Water: 1.36 US-Dollar
  - Shake Shack: 5.17 US-Dollar, 3.27 US-Dollar
- Sight seeing:
  - Statue Cruises (go to the statue of liberty): 18.00 US-Dollar
  - American Museum of Natural History: 22.00 US-Dollar
  - International Center of Photography: 10.00 US-Dollar
  - Phantom of the Opera: 48.57 Euro
miscellaneous
- Post stamps “Global Forever”: 6.90 US-Dollar
- Spider Killer spray and water: 3.95 US-Dollar
- GetGo: Water, 24Pack: 3.99 US-Dollar
- public transportation: 20 US-Dollar in total for my complete stay in Pittsburgh
- Cinema (Let’s be Cops, 16.08.2014): 10.25 US-Dollar

CMU stuff

When you come here, you can choose an AndrewID. This can be anything (letters).

WLAN Linux with “AddTrust External CA Root”

Terms

22 C.F.R. Part 62: A law regulating J1 visa.
ACH: Automated Clearing House, an electronic banking network often used for direct deposit and electronic bill payment
DS-2019: a form you need to apply for J-1 visa
GHC: Gates Hillman Center, a building
J-1 visa: non-immigrant visa issued by the United States to exchange visitors participating in programs that promote cultural exchange, especially to obtain medical or business training within the U.S.
OIE: Office of International Education
SEVIS: Student and Exchange Visitor Program (a program within U.S. Immigration and Customs Enforcement which monitors students and exchange visitors in the United States with F, M, or J visas status.)
USCIS: United States Citizenship and Immigration Services - a component of the United States Department of Homeland Security
MRV - Machine Readable Visa

Resources

interact.anthropomatik.kit.edu: Official interACT website
cmu.edu/oie
- Settling in Guide - very helpful!
StackExchange:
http://nutmeggle.wordpress.com/2013/06/23/the-welcome-pack-and-the-visa/

One-on-One Tutoring

info@martin-thoma.de (Martin Thoma) — Wed, 10 Sep 2014 10:03:00 +0200

I have heard that the famous universities in England have One-on-One tutoring. I’m not sure if that is actually the case, but that doesn’t matter for the idea I want to explain. That means one tutor helps one student only. That must be incredibly effective for learning. From my experience as a tutor and as a “coach” (Nachhilfelehrer) I know that there is a huge difference in One-on-One tutoring compared to One-on-Many tutoring, where one tutor teaches about 30 students. If you only have one student, you can focus on the needs of this single student. And I guess that the student will also be encouraged a bit more to participate. If you only have one student, the student will not sleep while you teach him. He has to stay focused, because you will notice when he doesn’t.

So One-on-One tutoring is something we want to get.

The problem of One-on-One tutoring

It is expensive, if it would be done like tutoring is currently done. In fact I think it would be impossible, because you would not find that many tutors. However, I think there are other ways to motivate students that have already passed the exam to be a tutor for another student.

How to get One-on-One tutoring

Give One-on-One tutors the possibility to write the exam again and improve their grade.

I could imagine it to work like this: A student passes the exam, but he does not have the grade he wants. As he has already passed the exam, he will not be allowed to simply write it again. So he has to search a student who hasn’t already passed the exam and help this student to prepare for the exam. Now we have to check if the One-on-One tutor really helped the new student. I could imagine two ways that could also be combined.

One-on-One tutoring: Way 1

The One-on-One tutor has to help the new student with the exercises. So the One-on-Many tutor gets the information which exercises were done by the One-on-One tutor and the new student. In contrast to now, not only one student would exercise, but two. In order to make sure that there is an improvement, one could increase the necessary number of points for the One-on-One tutor to a threshold of $\theta_1$ . $\theta_1$ should denote how much more points the student needs (in percent of the points the students need) to get the possibility to re-do the exam.

$\theta_1 = 0$ would mean that the tutor does only need a student who gets the minimum number of points to pass the exam.

$\theta_1 = 1$ would mean that the tutor needs his students to get double the number of points in the exercises than what he would need to be allowed to participate in the exam.

This means $\theta_1 \in [0, 1]$ .

$\theta_1$ should not be zero, because that would mean that 1-on-1 tutors could choose their students right before the exam. But I want them to choose the student at the beginning of the semester. And $\theta_1$ may not be too high. The higher it gets, the more difficult and time intensive it gets for the student and the tutor. I think the time right before an exam is the time where students learn most. If a student doesn’t achieve the $\theta_1$ threshold, then the tutor will not have any incentive to help the student to prepare for the exam except for money or friendship. This is a good incentive for the student to keep working on excercises, even if he already got enough points to be allowed to write the exam.

One-on-One tutoring: Way 2

The other possibility would be to enforce a minimum grade for the tutored student. So the One-on-One tutor cannot participate in the same exam as his student, but in the exam after that. And the tutor can only participate in that exam if his student got a minimum grade $\theta_2$ , e.g. 2.3 or better.

$\theta_2 = 5$ would mean that the student only has to participate in the exam (at KIT every student gets at least a grade of 5. The best grade is 1.0.).

$\theta_2 = 1$ would mean that the student has to pass with the best grade so that the tutor is allowed to participate in the exam.

Alternatively one could make that dependent on the number of students who got the grade. So $\theta_2 = 10\%$ would mean that you choose the grade threshold in a way that makes sure that at least 10% of all students pass that requirement.

This means $\theta_2 \in [1, 5]$ or $\theta_2 \in (0\%, 100\%]$ .

$\theta_2$ should not be too bad (near 5), because then it will be too easy for many students to get another try for the exam. That would also not help the students, because the tutor would not have an incentive that is strong enough to increase the abilities of his student. However, if $\theta_2$ gets too low (near 1.0) then most students will not try to be a tutor because it is too difficult to get a student to achieve that good results.

How to speak about it

I would like to call this model MOOT (Martins One-on-One Tutoring model ☺ ). It is parameterized with two variables, $\theta_1$ and $\theta_2$ .

I think $MOOT(\theta_1=0.2, \theta_2=25\%)$ would be a good possibility to encourage students to help other students effectively. $\theta_1$ is low enough to make it fairly easy for somebody who already passed the exam to “lift” a student to pass the requirement.

So I guess $MOOT(0.2, 25\%)$ would be a good choice. You could still argue that the tutor could simply write the excercises himself. But I don’t think that would be a problem, because then the tutor would do the excercises. He would still practice. We only get a problem if larger groups of students copy excercises. I don’t think that is the case, but I judge from the students I know. I might have a biased view on the problem.

Conclusion

The two ways I proposed would in general lead to better students, remove stress from students because a single bad exam would not be that bad any more and very likely lead to better students. Students would be better on the paper (because the tutor should only be able to improve himself by the new exam and also his student should be able to achieve better results in the exam) and in the real world. Students would get an incentive to repeat what they did not understand before. They would get an incentive to help other students. For free. And it would not be that much more work for the university. I think it is easier to correct exams of good students and only a few would really take this possibility as it is really time intensive for the One-on-One tutor. But for some exams and some students that might be worth the effort.

What do you think about it? Do you think we could introduce that model at KIT? Who could propose it? To whom should it be proposed?

Other incentives

If writing the exam again is not an option, we could try to find other incentives for tutors:

Getting “exclusive” rights:
- Visiting CERN or something similar
- Getting access to equipment of the university (executing your personal projects on KIT clusters, getting access to microscopes, being allowed to use lecture halls)
“Proud” of the unversity: If we could make KIT students being proud of being a student at KIT the students themselves could want to support fellow students.

I am not sure if there is any incentive which is strong enough to support another student for one semester. Do you have an idea? Please share it in the comments!

The Twiddle Algorithm

info@martin-thoma.de (Martin Thoma) — Sat, 06 Sep 2014 13:49:00 +0200

Twiddle is an algorithm that tries to find a good choice of parameters $p$ for an algorithm $\mathcal{A}$ that returns an error.

The algorithm is quite simple to implement. I guess gradient descent might be better for most cases, but Twiddle does not require any knowledge about the algorithm $\mathcal{A}$ which might be a big advantage. And you don’t have to calculate the gradient of high dimensional functions, which is nice, too.

The algorithm

Here is some pythonic pseudo code. A is an algorithm that returns an error.

# Choose an initialization parameter vector
p = [0, 0, 0]
# Define potential changes
dp = [1, 1, 1]
# Calculate the error
best_err = A(p)

threshold = 0.001

while sum(dp) > threshold:
    for i in range(len(p)):
        p[i] += dp[i]
        err = A(p)

        if err < best_err:  # There was some improvement
            best_err = err
            dp[i] *= 1.1
        else:  # There was no improvement
            p[i] -= 2*dp[i]  # Go into the other direction
            err = A(p)
 
            if err < best_err:  # There was an improvement
                best_err = err
                dp[i] *= 1.05
            else  # There was no improvement
                p[i] += dp[i]
                # As there was no improvement, the step size in either
                # direction, the step size might simply be too big.
                dp[i] *= 0.95

Hyperlapse

info@martin-thoma.de (Martin Thoma) — Wed, 03 Sep 2014 04:31:00 +0200

Johannes Kopf, a researcher at Microsoft, has recently published a stunning video about a software project he seems to have participated in. The video is about the automatic creation of hyperlapse videos:

We present a method for converting first-person videos, for example, captured with a helmet camera during activities such as rock climbing or bicycling, into hyperlapse videos: time-lapse videos with a smoothly moving camera.

So we are now only speaking about first-person videos. As videos created by a helmet camera might be very long and (as he accurately described it) “dead boring”, you want to speed that up. A timelaps would be a subsampling to every n-th frame. Those might be very shaky and hard to watch. That means you would at least want some image stabilization.

Microsoft Hyperlapse

But the software they created does more. Much more.

Reconstruction

First, it reconstructs the geometry of the captured environment. That alone is absolutely awesome. They apply a group of techniques called structure from motion.

Path planning

Then they plan the path of the camera. This is a 6-dimensional problem for every point in time where you want to get an image. The 6 dimensions are:

(x, y, z): Position of the camera
roll: horizontal, vertical angle
pitch: up, down view angle
yaw: left, right view angle

In case you have a problem with imagining roll, pitch and yaw you should take a look at the following image:

Roll, pitch and yaw
By [ZeroOne](https://commons.wikimedia.org/wiki/File:Flight_dynamics_with_text.png)

The chosen path should meet several criteria:

It should be smooth in space
Every path position should be close to input positions
The path should be short
Rotation should be smooth (that is what makes videos “shaky”)
The rendering quality should be as high as possible

The first 3 steps were achived by spline fitting.

Rendering

That step combines several output frames to render the desired camera image.

Nice timelapses

Mouse clicking games

info@martin-thoma.de (Martin Thoma) — Sat, 23 Aug 2014 14:15:00 +0200

Do you know games like cookie clicker where you only have to click a lot?

Cookie Clicker

You can do that automatically with the following Python script.

Just execute xdotool getmouselocation --shell before that to find the position of your mouse.

#!/usr/bin/env python

"""Tool to make automatic clicks VERY fast (useful for idle games)."""

import os
import time
import random


def main(clicks, twiggle, x, y, delay):
    for i in range(clicks):
        xrand = random.randint(-twiggle, twiggle)
        yrand = random.randint(-twiggle, twiggle)
        xpos, ypos = x+xrand, y+yrand
        os.system("xdotool mousemove %i %i click 1" % (xpos, ypos))
        if delay > 0:
            time.sleep(delay)


if __name__ == '__main__':
    from argparse import ArgumentParser

    parser = ArgumentParser()

    # Add more options if you like
    parser.add_argument("-c", "--clicks", dest="clicks",
                        help="number of clicks that will be done",
                        default=500,
                        type=int)
    parser.add_argument("-t", "--twiggle", dest="twiggle",
                        help="how much should the cursor move randomly",
                        default=50,
                        type=int)
    parser.add_argument("-x", dest="x",
                        help="x coordinate where to click",
                        default=711,
                        type=int)
    parser.add_argument("-y", dest="y",
                        help="y coordinate where to click",
                        default=467,
                        type=int)
    parser.add_argument("--delay", dest="delay",
                        help="delay between clicks",
                        default=0.01,
                        type=float)
    args = parser.parse_args()
    main(args.clicks, args.twiggle, args.x, args.y, args.delay)

Cookie Clicker after 5 minutes with a script

Have fun playing those games now!

Citizen Science Projects

info@martin-thoma.de (Martin Thoma) — Thu, 21 Aug 2014 13:27:00 +0200

“Citizen Science Projects” are research projects that crowdsource a part of the research work. The idea behind that is quite simple: Some tasks of researchers are very simple. Everybody can do them.

I think some of them are a great example of Gamification.

Galaxy Zoo

Galaxy Zoo is a crowdsourced astronomy project which invites people to assist in the classification of galaxies.

You get some images of Galaxies and you should see some characteristics by looking at them. You get about three possible answers to every question.

It looks like that:

Galaxy Zoo

Moon Zoo

High-resolution images of the Moon’s surface provided by the Lunar Reconnaissance Orbiter are used by volunteers to create detailed crater counts, mapping the variation in age of lunar rocks.

Source: Wikipedia

It looks like this:

Moon Zoo

EteRNA

EteRNA is a browser based game, developed by scientists at Carnegie Mellon University and Stanford University, that engages users to solve puzzles related to the folding of RNA molecules.

It looks like this:

EteRNA

Write Math

write-math.com is my own project. The aim of the project is to get fast and accurate recognition of mathematical symbols. To do so, I needed data.

Currently, the project is still under heavy development. Currently, most work I do is done offline. Hence the project might not improve until end of October (2014).

It looks like this:

write math

The most interesting part might be the interactive preprocessing experiments.

GPUs - Supercomputers for your home

info@martin-thoma.de (Martin Thoma) — Wed, 20 Aug 2014 23:26:00 +0200

A few days ago I got some of my neural net code to work with a GPU. The GPU is called “Tesla C2075”. It is able to get 515 GFlops peak performance. It has 448 CUDA cores that work with 1.15 GHz and it has 6GB GDDR5 memory.

My code needed about 10 hours to run before. After that, it only needed 10 minutes. That is 60 times faster! The library that did this miracle for me is called Theano)

Out of curiosity, I’ve searched for current high-end gamer graphic cards. I found nVidia Titan Z:

The Titan Z has 5760 CUDA cores. It can get 4061 GFLOPS x2 and has 12 GB of memory. That technological wonder-work costs only 2802 Euro.

To put that into perspective: In 2005, you would probably have been on place 68 of the TOP500 supercomputers world wide! (source).

Isn’t that crazy?

MediaViewer and Superprotect

info@martin-thoma.de (Martin Thoma) — Fri, 15 Aug 2014 14:35:00 +0200

Recently, a heated discussion started on the German Wikipedia about Superprotect. This article should give a very short summary how it came to this discussion.

Timeline

25. July 2014: Straw poll in the German Wiki began (link)
08. August 2014: Straw poll in the German Wiki ended. 72.5% voted for a default deactivation of the MediaViewer
09. /10. August 2014: The Admins DaB., Raymond and JEissfeldt (WMF) repeatedly change MediaWiki:Common.js to activate / deactivate the MediaViewer for the German wiki
10. August 2014: Superprotect got introduced and Erik Möller (User:Eloquence) protected MediaWiki:Common.js.

The MediaViewer

You can try the MediaViewer to get a feeling what this is all about.

You can view some images of the MediaViewer here:

Article
First click on image with MediaViewer
First click on image without MediaViewer
notes to the MediaViewer interface
MediaViewer Bottom 'Tab'
MediaViewer 'Use this file'

There has been a survey that suggests that 60% of all users like the MediaViewer (source). However, there are concerns that the survey was biased (source).

You should know that you can easily disable the MediaViewer if you have an account:

Preferences → Appearance → Files = uncheck ‘Enable Media Viewer’

The question is: Should the MediaViewer be activated per default?

Pro

Average users want to see the image in a higher resolution. The MediaViewer looks more professional than the old view.

Wrong arguments

The copyright notice doesn’t get in my way.
It is a legal obligation to show the copyright notice.

Contra

The MediaViewer doesn’t give new functionality: One could see high-resolution images before by simply clicking a second time on the image.
The MediaViewer makes it more difficult for new users to edit file descriptions.
The MediaViewer hides information like the copyright status.
- Charts and maps that are color-coded and change/are edited over time and have a legend/key in multiple languages simply do not work with this interface. See this example.
The MediaViewer makes browsing difficult on touch-screen devices that use pinch-to-zoom.
It becomes almost unusably slow on slower machines or machines on a slow connection.
MediaViewer makes it more difficult to find the image in the highest resolution.

Wrong arguments

It’s in the way of long-term wikipedia editors.
This argument is wrong, because long-term wikipedia editors can simply disable it. They should know how to do it.
Logged in in users don’t need this feature.
Again, they can disable it.

Good comments

The default setting for logged-in users should be the same as for unregistered users, so that new users have fewer surprises to deal with.

Alternatives to default-enable MediaViewer

Change from opt-out to opt-in
Disabled for current users and enabled for newly registered users

Superprotect

The German Wikipedia community decided to disable MediaViewer per default. As the MediaViewer got rolled out, some admins in the German Wikipedia enabled / disabled it (see version history of Common.js). This was the reason to create superprotect:

Add a new protection level called “superprotect”
Assigned to nobody by default. Requested by Erik Möller for the purposes of protecting pages such that sysop permissions are not sufficient to edit them. Change-Id: Idfa211257dbacc7623d42393257de1525ff01e9e

Source: gerrit.wikimedia.org

That started another community poll in the German wiki:

Wikipedia:Umfragen/Superschutz

and

Wikipedia:Meinungsbilder/Gruppenrecht Superschutz

Now the question is: Should the new group right ‘super protect’ be kept?

Pro

Super protect is a necessary tool to temporarily prevent abuse of admin powers in heated discussions.
The WMF has the legal right to do what they want on wikipedia.org.
The WMF controls the technical details of the Wikipedia platform, whereas the community only controls the content.

Wrong arguments

The super protect right was necessary as one admin was abusing his power.
If the problem is only one admin, there already is the possibility to remove the admin status of this user.
The WMF has the legal right to do what they want and they need superprotect to have legal security.
That is simply wrong. There is (currently) no case where an admin tried to put something illegal on Wikipedia.org and abused his admin rights to keep it there.

Contra

There is no need for such a tool, as the community is able to solve ‘wheel wars’.
In case of the abuse of powers by admins, these powers can be removed (see Review and removal of adminship)
The community wants to organize Wikipedia by themselves without intervention from the WMF. The problem for many users seems to be that the WMF is not elected, whereas admins are (see Wikipedia:Administrators#Becoming_an_administrator - there seem to be differences in the English / German Wiki).
The new concept of ‘super protection’ was introduced too fast without a concept which was discussed within the community.
The risk of misuse of power is too high as there is no mutual control.

Comments

I think super protect might be a good solution to temporarily freeze pages when more than two admins have a wheel war. In that case it could be implemented so that it can only freeze a page for 7 days and after those 7 days the page cannot be frozen for at least 14 days (one would have to discuss the numbers).

This way, it can be a tool for de-escalation and the abuse can be limited.

Also, sites that get frozen must have at least two back-and-forth edits by two admins and the community must have voted for freezing.

Terms

Wikipedia is the free Internet encyclopedia at wikipedia.org.

MediaWiki is the software that Wikipedia uses. It is also free and it gets developed by the Wikimedia Foundation and others. You can download MediaWiki here and find the source code via https://gerrit.wikimedia.org/r/p/mediawiki/core.git.

Wikimedia Foundation (short: WMF) is an American non-profit and charitable organization headquartered in San Francisco, California, that operates wikipedia.org.

MediaViewer is a JavaScript that gives the possibility to browse through all images of an article by using a diashow.

Wheel war is happening when two or more admins repeatedly revert their changes.

Sources

Start long running processes via SSH

info@martin-thoma.de (Martin Thoma) — Mon, 11 Aug 2014 14:51:00 +0200

Screen

screen is a nice tool that can be used to detach long running processes from the current SSH session - and be able to get it again!

Basic usage

You start it with

$ screen

You detach it with Ctrl + a and then d. After you pressed this key combination, you will see

[detached]

You get it back again with:

$ screen -r

Named sessions

You can start a named session with

$ screen -S foo

and get it back with

$ screen -r foo

nohup

$ nohup command &

William Shatner

info@martin-thoma.de (Martin Thoma) — Wed, 06 Aug 2014 07:42:00 +0200

William Shatner is the actor that played James Tiberius Kirk in the science fiction television series Star Trek. You can imagine how famous he is from these Twitter answers:

.@WilliamShatner Everything within normal parameters, Capt'n. Robotic probe @ESA_Rosetta arrives at comet #67P today! http://t.co/f7nuOjMVuh
— ESA (@esa) 6. August 2014

@WilliamShatner Good day, Captain. #ISS is in standard orbit and Commander Swanson has the conn. Hope you’re having a great weekend!
— NASA (@NASA) 2. August 2014

Logging in Python

info@martin-thoma.de (Martin Thoma) — Fri, 01 Aug 2014 21:15:00 +0200

Python has a nice logging module. You can use it like this:

Stream output

#!/usr/bin/env python

import logging
import sys

logging.basicConfig(format='%(asctime)s %(levelname)s %(message)s',
                    level=logging.DEBUG,
                    stream=sys.stdout)
logging.debug('This message should go to the log file')
logging.info('So should this')
logging.warning('And this, too')

File output

#!/usr/bin/env python

import logging

logging.basicConfig(filename='logging.log',
                    format='%(asctime)s %(levelname)s %(message)s',
                    level=logging.DEBUG)
logging.debug('This message should go to the log file')
logging.info('So should this')
logging.warning('And this, too')

Configuration files in Python

info@martin-thoma.de (Martin Thoma) — Sun, 27 Jul 2014 15:46:00 +0200

Most interesting programs need some kind of configuration:

Content Management Systems like WordPress blogs, WikiMedia and Joomla need to store the information where the database server is (the hostname) and how to login (username and password)
Proprietary software might need to store if the software was registered already (the serial key)
Scientific software could store the path to BLAS libraries

For very simple tasks you might choose to write these configuration variables directly into the source code. But this is a bad idea when you upload the code to GitHub.

I will explain some alternatives I got to know for Python.

Python Configuration File

The simplest way to write configuration files is to simply write a separate file that contains Python code. You might want to call it something like databaseconfig.py. Then you could add the line *config.py to your .gitignore file to avoid uploading it accidentally.

A configuration file could look like this:

#!/usr/bin/env python
import preprocessing
mysql = {'host': 'localhost',
         'user': 'root',
         'passwd': 'my secret password',
         'db': 'write-math'}
preprocessing_queue = [preprocessing.scale_and_center,
                       preprocessing.dot_reduction,
                       preprocessing.connect_lines]
use_anonymous = True

Within the actual code, you can use it like this:

#!/usr/bin/env python
import databaseconfig as cfg
connect(mysql['host'], mysql['user'], mysql['password'])

The way you include the configuration might feel very convenient at a first glance, but imagine what happens when you get more configuration variables. You definitely need to provide an example configuration file. And it is hard to resist the temptation to include code within the configuration file.

JSON

JSON is short for JavaScript Object Notation. It is widespread and thus has good support for many programming languages.

The configuration might look like this:

{
    "mysql":{
        "host":"localhost",
        "user":"root",
        "passwd":"my secret password",
        "db":"write-math"
    },
    "other":{
        "preprocessing_queue":[
            "preprocessing.scale_and_center",
            "preprocessing.dot_reduction",
            "preprocessing.connect_lines"
            ],
        "use_anonymous":true
    }
}

You can read it like this:

import json

with open('config.json') as json_data_file:
    data = json.load(json_data_file)
print(data)

which outputs

{u'mysql': {u'db': u'write-math',
            u'host': u'localhost',
            u'passwd': u'my secret password',
            u'user': u'root'},
 u'other': {u'preprocessing_queue': [u'preprocessing.scale_and_center',
                                     u'preprocessing.dot_reduction',
                                     u'preprocessing.connect_lines'],
            u'use_anonymous': True}}

Writing JSON files is also easy. Just build up the dictionary and use

import json
with open('config.json', 'w') as outfile:
    json.dump(data, outfile)

YAML

YAML is a configuration file format. Wikipedia says:

YAML (rhymes with camel) is a human-readable data serialization format that takes concepts from programming languages such as C, Perl, and Python, and ideas from XML and the data format of electronic mail (RFC 2822). YAML was first proposed by Clark Evans in 2001, who designed it together with Ingy döt Net and Oren Ben-Kiki. It is available for several programming languages.

The file itself might look like this:

mysql:
    host: localhost
    user: root
    passwd: my secret password
    db: write-math
other:
    preprocessing_queue:
        - preprocessing.scale_and_center
        - preprocessing.dot_reduction
        - preprocessing.connect_lines
    use_anonymous: yes

You can read it like this:

import yaml

with open("config.yml", 'r') as ymlfile:
    cfg = yaml.load(ymlfile)

for section in cfg:
    print(section)
print(cfg['mysql'])
print(cfg['other'])

It outputs:

other
mysql
{'passwd': 'my secret password',
 'host': 'localhost',
 'db': 'write-math',
 'user': 'root'}
{'preprocessing_queue': ['preprocessing.scale_and_center',
                         'preprocessing.dot_reduction',
                         'preprocessing.connect_lines'],
 'use_anonymous': True}

There is a yaml.dump method, so you can write the configuration the same way. Just build up a dictionary.

YAML is used by the Blender project.

Resources

Documentation

INI

INI files look like this:

[mysql]
host=localhost
user=root
passwd=my secret password
db=write-math

[other]
preprocessing_queue=["preprocessing.scale_and_center",
                     "preprocessing.dot_reduction",
                     "preprocessing.connect_lines"]
use_anonymous=yes

ConfigParser

Basic example

The file can be loaded and used like this:

#!/usr/bin/env python

import ConfigParser
import io

# Load the configuration file
with open("config.yml") as f:
    sample_config = f.read()
config = ConfigParser.RawConfigParser(allow_no_value=True)
config.readfp(io.BytesIO(sample_config))

# List all contents
print("List all contents")
for section in config.sections():
    print("Section: %s" % section)
    for options in config.options(section):
        print("x %s:::%s:::%s" % (options,
                                  config.get(section, options),
                                  str(type(options))))

# Print some contents
print("\nPrint some contents")
print(config.get('other', 'use_anonymous'))  # Just get the value
print(config.getboolean('other', 'use_anonymous'))  # You know the datatype?

which outputs

List all contents
Section: mysql
x host:::localhost:::<type 'str'>
x user:::root:::<type 'str'>
x passwd:::my secret password:::<type 'str'>
x db:::write-math:::<type 'str'>
Section: other
x preprocessing_queue:::["preprocessing.scale_and_center",
"preprocessing.dot_reduction",
"preprocessing.connect_lines"]:::<type 'str'>
x use_anonymous:::yes:::<type 'str'>

Print some contents
yes
True

As you can see, you can use a standard data format that is easy to read and write. Methods like getboolean and getint allow you to get the datatype instead of a simple string.

Writing configuration

import os
configfile_name = "config.yaml"

# Check if there is already a configurtion file
if not os.path.isfile(configfile_name):
    # Create the configuration file as it doesn't exist yet
    cfgfile = open(configfile_name, 'w')

    # Add content to the file
    Config = ConfigParser.ConfigParser()
    Config.add_section('mysql')
    Config.set('mysql', 'host', 'localhost')
    Config.set('mysql', 'user', 'root')
    Config.set('mysql', 'passwd', 'my secret password')
    Config.set('mysql', 'db', 'write-math')
    Config.add_section('other')
    Config.set('other',
               'preprocessing_queue',
               ['preprocessing.scale_and_center',
                'preprocessing.dot_reduction',
                'preprocessing.connect_lines'])
    Config.set('other', 'use_anonymous', True)
    Config.write(cfgfile)
    cfgfile.close()

results in

[mysql]
host = localhost
user = root
passwd = my secret password
db = write-math

[other]
preprocessing_queue = ['preprocessing.scale_and_center', 'preprocessing.dot_reduction', 'preprocessing.connect_lines']
use_anonymous = True

XML

Seems not to be used at all for configuration files by the Python community. However, parsing / writing XML is easy and there are plenty of possibilities to do so with Python. One is BeautifulSoup:

from BeautifulSoup import BeautifulSoup

with open("config.xml") as f:
    content = f.read()

y = BeautifulSoup(content)
print(y.mysql.host.contents[0])
for tag in y.other.preprocessing_queue:
    print(tag)

where the config.xml might look like this:

<config>
    <mysql>
        <host>localhost</host>
        <user>root</user>
        <passwd>my secret password</passwd>
        <db>write-math</db>
    </mysql>
    <other>
        <preprocessing_queue>
            <li>preprocessing.scale_and_center</li>
            <li>preprocessing.dot_reduction</li>
            <li>preprocessing.connect_lines</li>
        </preprocessing_queue>
        <use_anonymous value="true" />
    </other>
</config>

File Endings

File Endings give the user and the system an indicator about the content of a file. Reasonable file endings for configuration files are

*config.py for Python files
*.yaml or *.yml if the configuration is done in YAML format
*.json for configuration files written in JSON format
*.cfg or *.conf to indicate that it is a configuration file
*.ini for “initialization” are quite widespread (see Wiki)
~/.[my_app_name]rc is a VERY common naming scheme for configuration files on Linux systems. RC is a reference to an old computer system and means “run common”.

That said, I think I prefer *.conf. I think it is a choice that users understand.

But you might also consider that *.ini might get opened by standard in a text editor. For the other options, users might get asked which program they want to use.

Resources

Append Python PATH

info@martin-thoma.de (Martin Thoma) — Sun, 20 Jul 2014 14:04:00 +0200

Python has a PATH in which it looks for modules. You can display the current module PATH with

import sys
print(sys.path)

and apped something to it with

import sys
sys.path.append("/some/path/to/a/module")

However, the clean way would be to write a module and install that module.

Introduction to Octave

info@martin-thoma.de (Martin Thoma) — Fri, 18 Jul 2014 17:40:00 +0200

GNU Octave is a really neat prototyping language for machine learning tasks. It is dynamically typed.

Installation

Octave is in the package repositories, so it can be installed by

$ sudo apt-get install octave gnuplot-x11 octave-epstk

and started with

$ octave

Configuration

Create a file ~/.octaverc in your home folder. Write

PS1('>>');
setenv("GNUTERM","x11");

in it to get a nicer prompt and make sure that plots will work.

The Language

Vectors and Matrices

Octave has a lot of neat matrix manipulation features. You can create a matrix $A = \begin{pmatrix} 1 & 2\\ 3 & 4\end{pmatrix}$ with

>> A = [1 2; 3 4];

When you want to transpose a vector / a matrix, you simply add an apostrophe:

>> A = [1 2; 3 4];
>> A'
ans =

   1   3
   2   4

You can multiply two matrices with A*B or use the dot product with A .* B.

The identity matrix $I \in \mathbb{R}^{n \times n}$ can be created with

>> I = eye(n);

You can get a part of the matrix by slicing:

>> I = eye(n);
>> I(:, 1:2);

But be careful: Vectors and matrices are 1-indexed, not 0-indexed as you might expect!

You can get the size of a matrix with the function size which returns a matrix:

>> a = [1 2 3; 4 5 6];
>> size(a)
ans =

   2   3

If you simple want the “length” you can directly access the first element:

>> size(a)(1)
ans =  2

Sequences

The sequence 0 1 2 3 4 5 can be created with [0:5].

The sequence 0.2 0.3 0.4 0.5 can be created with [0.2:0.1:0.5]. In general: [<start>:<step>:<end>] where <start> and <end> are included.

You can also very simple apply functions to each element:

>> t = [0.2:0.1:0.5];
>> sin(t)
ans =

   0.19867   0.29552   0.38942   0.47943

The output can be suppressed with ;.

Plotting

I have never seen a language where plotting is so easy:

>> x = [0:0.01:pi];
>> y = sin(x);
>> plot(x, y);

You can add labels and a legend to it, too:

>> xlabel = "x";
>> ylabel = "value";
>> legend('sin', 'cos')
>> title("sin and cos")

And finally, you can store the image:

print -dpng 'my_plot.png'

Control statements

for

for i=1:10;
    printf("%i: %i\n", i, i^2)
end

while

i=1;
while i <= 10,
    printf("%i: %i\n", i, i^2)
end;

if

if 2 == 1+1,
    printf("True\n");
elseif 3 == 2+1,
    printf("Else true");
else
    printf("else");
end;

Functions

Functions have to be saved in a file called [filename].m. One other special thing about functions is that you define the variable with the output at the beginning:

function y = fibonacci(n)
    if n < 2,
        y = 1;
    else
        y = fibonacci(n-1) + fibonacci(n-2);
    end;

You can also group values you want to give back like this:

function [succ, pred] = succ_and_pred(n)
    succ = n+1;
    pred = n-1;

Resources

Fix zypper readline error

info@martin-thoma.de (Martin Thoma) — Mon, 14 Jul 2014 11:34:00 +0200

In case you work on a openSUSE system and you get the following error

$ zypper
zypper: symbol lookup error: /usr/lib/libreadline.so.6: undefined symbol: PC

you can probably “fix” it by setting the 64 Bit LD_LIBRARY like this:

$ export LD_LIBRARY_PATH=/lib64:$LD_LIBRARY_PATH

It worked for me on openSUSE 12.1 “Asparagus”.

Credits

Thanks to JRSETI’s Blog

Change password in ATIS

info@martin-thoma.de (Martin Thoma) — Sat, 12 Jul 2014 17:24:00 +0200

Recently, the Heartbleed bug was discovered. It works like this:

Heartbleed Explanation
From xkcd

One effect of that bug is that you have to change your password.

From xkcd

You can do that from home via SSH. Just replace s_thoma by s_[your last name]:

$ ssh s_thoma@i08fs1.ira.uka.de
s_thoma@i08fs1.ira.uka.de's password: 
s_thoma@i08fs1(~)$ passwd
Changing password for user s_thoma.
Enter login(AD) password: 
Current Password: 
New password: 
Retype new password: 
AD password information changed for s_thoma
passwd: all authentication tokens updated successfully.
s_thoma@i08fs1(~)$ exit
logout
Connection to i08fs1.ira.uka.de closed.

In case you don’t remember your password: Take a look at your Browser settings. You’ve probably entered it already for looking at your printing account and eventually your browser saved it.

A.I. in Computer Games

info@martin-thoma.de (Martin Thoma) — Tue, 01 Jul 2014 23:52:00 +0200

Artificial Intelligences (A.I.s) are computer programs that are able to adjust their behaviour according to data they see. So A.I.s are able to adjust to the data a human player generates.

Game A.I.s

Solved games

There is a number of games which are definitely solved. That means the A.I. plays perfectly:

Tic-Tac-Toe
Connect Four: A Knowledge-based Approach of Connect-Four. Amsterdam, 1988. Victor Allis.
Checkers:

Computers win always

A second category are games in which A.I.s always win against human players, but they don’t have a perfect strategy. Or at least we have not proven that they have a perfect strategy:

Chess
Go on a 5×5 board
Reversi on a 4×4 board

Update: There are advances on the 19×19 field:

Paper
Nature: Mastering the game of Go with deep neural networks and tree search
YouTube by nature: The computer that mastered Go
Google Blog: AlphaGo: using machine learning to master the ancient game of Go

Unspecialized Game A.I.s

The following video is an explanation and demo of software Tom Murphy VII wrote that learns how to play a Nintendo Entertainment System game and then automatically plays it. It’s called “learnfun” (for learn function).

You might want to skip to 6:13 for the demo:

Research paper published in SIGBOVIK 2013: “The first level of Super Mario Bros. is easy with lexicographic ordering a and time travel …after that it gets a little tricky.”

There is a follow-up video with Zelda, Punch-Out, Dr. Mario, Contra, Wall Street Kid and Russian Attack:

And a third episode with Super Mario, Gradius, Mega Man 2, Pro Wrestling, Color a Dinosaur, Nintendo Pinball, Cliffhanger, Arkanoid, Double Dare, Ice hockey:

Super Mario A.I. Competition

Linux Commands for Working from home

info@martin-thoma.de (Martin Thoma) — Mon, 30 Jun 2014 13:41:00 +0200

This article is just a collection of commands and shortcuts I need quite often.

Creating a single archive file from a complete folder

$ tar -zcvf ~/[target file].tar.gz [folder you want to copy]

Getting a file from a remote host to localhost

$ scp [username]@[host]:[path/to/remote/file] [path/to/local/folder]

Shell Shortcuts

Copy a selected text:

Ctrl + Shift + C

Paste a text from clipboard to the command line:

Ctrl + Shift + V

Copy a selected text from the command line and paste it:

Mouse wheel click

Search through the history of commands you have typed before:

Ctrl + R

Web stuff

### Watching the Apache error log

$ tail -f  /var/log/apache2/error.log

Truncating the Apache error log

$ sudo truncate -s 0 /var/log/apache2/error.log

Finding php.ini

$ php -i | grep "Loaded Configuration File"

Importing data to MySQL

$ mysql --host localhost -u root -p write-math < wm_raw_draw_data.sql

What are pfiles?

info@martin-thoma.de (Martin Thoma) — Fri, 27 Jun 2014 06:39:00 +0200

pfile is a binary file format that is used in ASR for storing feature vectors and their corresponding labels. This file format is sometimes also called ICSI feature file archive format. But this file format cannot be used for ASR only, but also for many other ML tasks.

The file consists of a fixed length ascii header followed by zero or more variable length binary sections. Each parameter in the header has a name and a list of zero or more value strings. The programmer’s interface to pfiles (see param.h and pfile.h) allows each parameter value to be interpreted as integer, float, string, arrays, distributed vector, matrix, mapping tables, etc.

Some special parameter names are associated with a section in the binary part of the pfile. The value strings for these parameters give the size and offset (from the end of the header) of the binary section.

A binary section can be used as a one dimensional sequence of values, or as a sequence of fixed length rows in a two dimensional matrix.

Some parameters are automatically added by the pfile command. For example, pfile_header is a parameter that contains the length and version number of the header.

Source: old-site.clsp.jhu.edu/ws96/ris/man/pfile.doc

pfile_utils

pfile_utils is a toolset to manage pfiles. It is part of the SPRACHcore software package. The project is located at code.google.com/p/pfile-utilities and seems to be in version 0.51 by now. The code is written in C++.

pfile_info gives general information about the file:

$ pfile_info all.pfile
all.pfile
9581 sentences, 3158027 frames, 1 label(s), 42 features

pfile_create

You can call pfile_create like this:

$ ./pfile_create -i - -f 1 -l 1 -o output.pfile
0 1 1
1 2 0
2 1 1
3 1 1
4 42 4
0 0 0
1 1337 0
2 2 2
0 3 3

You can end the input with Ctrl + D.

The numbers are:

[sentence-nr] [frame-nr] [feature 1] [feature 2] ... [feature n] [label 1] [label 2] ... [label n]

where the option -f defines the number of features and -l defines the number of labels. Please note that within one sentence, the number of frames has to be increasing by exactly one. One sentence can have an arbitrary number of frames, but as soon as you make another sentence, you need to increase this number by exactly one.

Awesome Robots

info@martin-thoma.de (Martin Thoma) — Sun, 22 Jun 2014 22:39:00 +0200

Robots are mechanical devices that are controlled by computer programs. Some of them act unexpectedly intelligent due to recent improvements in processing speed (hardware) and neural nets (computer science) as well as good application of differential equations (mathematics).

Differential equations are mathematical equations with a function and its derivative. So a simple differential equation is $f(x) = f'(x)$ . They are commonly used in physics, because you often have functions that have time as their argument. A simple example would be the growth of a population in biology (see Differential Equations: some simple examples from Physclips).

Cubli

The Cubli is a 15 × 15 × 15 cm cube that can jump up and balance on its corner. Reaction wheels mounted on three faces of the cube rotate at high angular velocities and then brake suddenly, causing the Cubli to jump up. Once the Cubli has almost reached the corner stand up position, controlled motor torques are applied to make it balance on its corner. In addition to balancing, the motor torques can also be used to achieve a controlled fall such that the Cubli can be commanded to fall in any arbitrary direction. Combining these three abilities – jumping up, balancing, and controlled falling – the Cubli is able to ‘walk’.

Quadcopters by Raffaello D’Andrea

In a robot lab at TEDGlobal, Raffaello D’Andrea demos his flying quadcopters: robots that think like athletes, solving physical problems with algorithms that help them learn. In a series of nifty demos, D’Andrea show drones that play catch, balance and make decisions together – and watch out for an I-want-this-now demo of Kinect-controlled quads.

Kuka Table Tennis Robot

The unbelievably fast KUKA robot faces off against one of the best table tennis players of all time. Who has the best technique? Who will win the first ever table tennis duel of human versus robot?

Watch this thrilling commercial of table tennis and robotics performed at the highest level. The KUKA KR AGILUS demonstrates its skills with the table tennis racket - a realistic vision of what robots can be capable of in the future.

Timo Boll, the German table tennis star, is the new brand ambassador for KUKA Robotics in China. The collaboration celebrates the inherent speed, precision, and flexibility of KUKA’s industrial robots in tandem with Boll’s electrifying and tactical prowess in competition. To celebrate the new KUKA Robotics factory in Shanghai, the thrilling video was a highlight of the Grand Opening on March 11th, 2014. The 20,000 sq. meter space will produce the KR QUANTEC series robot as well as the KRC4 universal controller for the Asian market. As a market leader in China, KUKA aims to further develop automation in the country while providing a modern and employee-friendly working environment.

More: kuka-timoboll.com

Stickybot

A gecko-like robot from Stanford.

Cheetah

A robot by Boston Dynamics that runs 45 km/h!

Snakebot

From the Biorobotics Lab at Carnegie Mellon University, a snake robot (Snakebot) demonstrates how it can climb a tree and look around.

Please keep in mind that this robot climbed a specific tree with a specific trunk width about 1 meter off of the ground. The researchers working to design, build and program these robots still have much work to do to get these bots to climb taller trees of various sizes and to navigate over branches and wires.

Kilobots

A thousand-robot swarm created by Harvard researchers can self-assemble into different shapes. Learn more: A Thousand Kilobots Self-Assemble Into Complex Shapes

Classify MNIST with PyBrain

info@martin-thoma.de (Martin Thoma) — Wed, 18 Jun 2014 02:31:00 +0200

The MNIST database is a huge database of handwritten digits that is commonly used for training, evaluating and comparing classifiers.

It has a training set of 60,000 instances and a test set of 10,000 instances. Every instance is a 28 × 28 pixel grayscale image.

The MNIST format

MNIST comes in 4 files (download here):

train-images-idx3-ubyte.gz: training set images (9.45 MB, 60000 instances)
train-labels-idx1-ubyte.gz: training set labels (28.2 kB, 60000 labels)
t10k-images-idx3-ubyte.gz: test set images (1.57 MB, 10000 instances)
t10k-labels-idx1-ubyte.gz: test set labels (4.43 kB, 10000 labels)

Both label files are like this:

[offset] [type]          [value]          [description] 
0000     32 bit integer  0x00000801(2049) magic number (MSB first) 
0004     32 bit integer  60000            number of items 
0008     unsigned byte   ??               label 
0009     unsigned byte   ??               label 
........ 
xxxx     unsigned byte   ??               label
The labels values are 0 to 9.

and both image containers are like that:

[offset] [type]          [value]          [description] 
0000     32 bit integer  0x00000803(2051) magic number 
0004     32 bit integer  60000            number of images 
0008     32 bit integer  28               number of rows 
0012     32 bit integer  28               number of columns 
0016     unsigned byte   ??               pixel 
0017     unsigned byte   ??               pixel 
........ 
xxxx     unsigned byte   ??               pixel

And they are, of course, compressed.

Reading the dataset

Python brings all necessary tools to make it easy to read the dataset:

gzip: A library for reading gzipped files
unpack to read the packed binary data

As the training and the testing dataset is structured the same way, we can create a method thad retrives the data for both files.

from struct import unpack
import gzip
from numpy import zeros, uint8


def get_labeled_data(imagefile, labelfile):
    """Read input-vector (image) and target class (label, 0-9) and return
       it as list of tuples.
    """
    # Open the images with gzip in read binary mode
    images = gzip.open(imagefile, 'rb')
    labels = gzip.open(labelfile, 'rb')

    # Read the binary data

    # We have to get big endian unsigned int. So we need '>I'

    # Get metadata for images
    images.read(4)  # skip the magic_number
    number_of_images = images.read(4)
    number_of_images = unpack('>I', number_of_images)[0]
    rows = images.read(4)
    rows = unpack('>I', rows)[0]
    cols = images.read(4)
    cols = unpack('>I', cols)[0]

    # Get metadata for labels
    labels.read(4)  # skip the magic_number
    N = labels.read(4)
    N = unpack('>I', N)[0]

    if number_of_images != N:
        raise Exception('number of labels did not match the number of images')

    # Get the data
    x = zeros((N, rows, cols), dtype=uint8)  # Initialize numpy array
    y = zeros((N, 1), dtype=uint8)  # Initialize numpy array
    for i in range(N):
        if i % 1000 == 0:
            print("i: %i" % i)
        for row in range(rows):
            for col in range(cols):
                tmp_pixel = images.read(1)  # Just a single byte
                tmp_pixel = unpack('>B', tmp_pixel)[0]
                x[i][row][col] = tmp_pixel
        tmp_label = labels.read(1)
        y[i] = unpack('>B', tmp_label)[0]
    return (x, y)

Viewing data

You might want to take a look at an image. This is easily possible with PyLab (which is a part of Matplotlib):

from pylab import imshow, show, cm


def view_image(image, label=""):
    """View a single image."""
    print("Label: %s" % label)
    imshow(image, cmap=cm.gray)
    show()

It tooks like this:

Classify data

Now we can use PyBrain to classify data.

The following code will first build the PyBrain datastructure for the training set and the testing set. Then it will build a very simple neural network called a Multilayer Perceptron (MLP) with three layers: An input layer, a hidden layer and an output layer.

After creating it, the MLP will be trained with the backpropagation algorithm. Every training step is followed by an evaluation step.

By the way, numpy.ravel simply flattens a list.

from numpy import ravel

from pybrain.datasets import ClassificationDataSet
from pybrain.utilities import percentError
from pybrain.tools.shortcuts import buildNetwork
from pybrain.supervised.trainers import BackpropTrainer
from pybrain.structure.modules import SoftmaxLayer


def classify(training, testing, HIDDEN_NEURONS, MOMENTUM, WEIGHTDECAY,
             LEARNING_RATE, LEARNING_RATE_DECAY, EPOCHS):
    trndata = ClassificationDataSet(INPUT_FEATURES, 1, nb_classes=CLASSES)
    tstdata = ClassificationDataSet(INPUT_FEATURES, 1, nb_classes=CLASSES)

    for i in range(len(testing)):
        tstdata.addSample(ravel(testing['x'][i]), [testing['y'][i]])
    for i in range(len(trndata)):
        trndata.addSample(ravel(trndata['x'][i]), [trndata['y'][i]])

    # This is necessary, but I don't know why
    # See http://stackoverflow.com/q/8154674/562769
    trndata._convertToOneOfMany()
    tstdata._convertToOneOfMany()

    fnn = buildNetwork(trndata.indim, HIDDEN_NEURONS, trndata.outdim,
                       outclass=SoftmaxLayer)

    trainer = BackpropTrainer(fnn, dataset=trndata, momentum=MOMENTUM,
                              verbose=True, weightdecay=WEIGHTDECAY,
                              learningrate=LEARNING_RATE,
                              lrdecay=LEARNING_RATE_DECAY)
    for i in range(EPOCHS):
        trainer.trainEpochs(1)
        trnresult = percentError(trainer.testOnClassData(),
                                 trndata['class'])
        tstresult = percentError(trainer.testOnClassData(
                                 dataset=tstdata), tstdata['class'])

        print("epoch: %4d" % trainer.totalepochs,
              "  train error: %5.2f%%" % trnresult,
              "  test error: %5.2f%%" % tstresult)
    return fnn

Final steps

You might want to add command line parameters, logging and probably pickle the data.

Final complete code

#!/usr/bin/env python

from struct import unpack
import gzip
from numpy import zeros, uint8, ravel

from pylab import imshow, show, cm

from pybrain.datasets import ClassificationDataSet
from pybrain.utilities import percentError
from pybrain.tools.shortcuts import buildNetwork
from pybrain.supervised.trainers import BackpropTrainer
from pybrain.structure.modules import SoftmaxLayer

from argparse import ArgumentParser
import os.path
import cPickle as pickle


def get_labeled_data(imagefile, labelfile, picklename):
    """Read input-vector (image) and target class (label, 0-9) and return
       it as list of tuples.
    """
    if os.path.isfile('%s.pickle' % picklename):
        data = pickle.load(open('%s.pickle' % picklename))
    else:
        # Open the images with gzip in read binary mode
        images = gzip.open(imagefile, 'rb')
        labels = gzip.open(labelfile, 'rb')

        # Read the binary data

        # We have to get big endian unsigned int. So we need '>I'

        # Get metadata for images
        images.read(4)  # skip the magic_number
        number_of_images = images.read(4)
        number_of_images = unpack('>I', number_of_images)[0]
        rows = images.read(4)
        rows = unpack('>I', rows)[0]
        cols = images.read(4)
        cols = unpack('>I', cols)[0]

        # Get metadata for labels
        labels.read(4)  # skip the magic_number
        N = labels.read(4)
        N = unpack('>I', N)[0]

        if number_of_images != N:
            raise Exception('The number of labels did not match '
                            'the number of images.')

        # Get the data
        x = zeros((N, rows, cols), dtype=uint8)  # Initialize numpy array
        y = zeros((N, 1), dtype=uint8)  # Initialize numpy array
        for i in range(N):
            if i % 1000 == 0:
                print("i: %i" % i)
            for row in range(rows):
                for col in range(cols):
                    tmp_pixel = images.read(1)  # Just a single byte
                    tmp_pixel = unpack('>B', tmp_pixel)[0]
                    x[i][row][col] = (float(tmp_pixel) / 255)
            tmp_label = labels.read(1)
            y[i] = unpack('>B', tmp_label)[0]
        data = {'x': x, 'y': y, 'rows': rows, 'cols': cols}
        pickle.dump(data, open("%s.pickle" % picklename, "wb"))
    return data


def view_image(image, label=""):
    """View a single image."""
    print("Label: %s" % label)
    imshow(image, cmap=cm.gray)
    show()


def classify(training, testing, HIDDEN_NEURONS, MOMENTUM, WEIGHTDECAY,
             LEARNING_RATE, LEARNING_RATE_DECAY, EPOCHS):
    INPUT_FEATURES = testing['rows'] * testing['cols']
    print("Input features: %i" % INPUT_FEATURES)
    CLASSES = 10
    trndata = ClassificationDataSet(INPUT_FEATURES, 1, nb_classes=CLASSES)
    tstdata = ClassificationDataSet(INPUT_FEATURES, 1, nb_classes=CLASSES)

    for i in range(len(testing['x'])):
        tstdata.addSample(ravel(testing['x'][i]), [testing['y'][i]])
    for i in range(len(training['x'])):
        trndata.addSample(ravel(training['x'][i]), [training['y'][i]])

    # This is necessary, but I don't know why
    # See http://stackoverflow.com/q/8154674/562769
    trndata._convertToOneOfMany()
    tstdata._convertToOneOfMany()

    fnn = buildNetwork(trndata.indim, HIDDEN_NEURONS, trndata.outdim,
                       outclass=SoftmaxLayer)

    trainer = BackpropTrainer(fnn, dataset=trndata, momentum=MOMENTUM,
                              verbose=True, weightdecay=WEIGHTDECAY,
                              learningrate=LEARNING_RATE,
                              lrdecay=LEARNING_RATE_DECAY)
    for i in range(EPOCHS):
        trainer.trainEpochs(1)
        trnresult = percentError(trainer.testOnClassData(),
                                 trndata['class'])
        tstresult = percentError(trainer.testOnClassData(
                                 dataset=tstdata), tstdata['class'])

        print("epoch: %4d" % trainer.totalepochs,
                     "  train error: %5.2f%%" % trnresult,
                     "  test error: %5.2f%%" % tstresult)
    return fnn

if __name__ == '__main__':
    parser = ArgumentParser()

    # Add more options if you like
    parser.add_argument("-H", metavar="H", type=int, dest="hidden_neurons",
                        default=200,
                        help="number of neurons in the hidden layer")
    parser.add_argument("-e", metavar="EPOCHS", type=int,
                        dest="epochs", default=20,
                        help="number of epochs to learn")
    parser.add_argument("-d", metavar="W", type=float, dest="weightdecay",
                        default=0.01,
                        help="weightdecay")
    parser.add_argument("-m", metavar="M", type=float, dest="momentum",
                        default=0.1,
                        help="momentum")
    parser.add_argument("-l", metavar="ETA", type=float, dest="learning_rate",
                        default=0.01,
                        help="learning rate")
    parser.add_argument("-ld", metavar="ALPHA", type=float, dest="lrdecay",
                        default=1,
                        help="learning rate decay")
    args = parser.parse_args()

    print("Get testset")
    testing = get_labeled_data('t10k-images-idx3-ubyte.gz',
                               't10k-labels-idx1-ubyte.gz', 'testing')
    print("Got %i testing datasets." % len(testing['x']))
    print("Get trainingset")
    training = get_labeled_data('train-images-idx3-ubyte.gz',
                                'train-labels-idx1-ubyte.gz', 'training')
    print("Got %i training datasets." % len(training['x']))
    classify(training, testing, args.hidden_neurons, args.momentum,
             args.weightdecay, args.learning_rate, args.lrdecay, args.epochs)

Results

I’ll update that tomorrow:

python pybrainmnist.py -e 10000 -H 300
Get testset
Got 10000 testing datasets.
Get trainingset
Got 60000 training datasets.
Input features: 784
Total error: 0.0473157733222
('epoch:    1', '  train error: 89.56%', '  test error: 89.72%')
Total error: 0.0471086288906
('epoch:    2', '  train error: 90.13%', '  test error: 90.20%')
Total error: 0.0471545599893
('epoch:    3', '  train error: 90.25%', '  test error: 90.26%')
Total error: 0.0471544804653
('epoch:    4', '  train error: 90.08%', '  test error: 89.91%')
Total error: 0.047137394504
('epoch:    5', '  train error: 90.25%', '  test error: 90.26%')
Total error: 0.0471630159397
('epoch:    6', '  train error: 89.56%', '  test error: 89.72%')
Total error: 0.0471585130872
('epoch:    7', '  train error: 89.78%', '  test error: 89.90%')
Total error: 0.0471376594558
('epoch:    8', '  train error: 90.14%', '  test error: 90.42%')
Total error: 0.0471355052763
('epoch:    9', '  train error: 90.97%', '  test error: 91.08%')
Total error: 0.0471595886207
('epoch:   10', '  train error: 90.07%', '  test error: 89.68%')
Total error: 0.0471580212242
('epoch:   11', '  train error: 90.13%', '  test error: 90.20%')
Total error: 0.047146194285
('epoch:   12', '  train error: 90.07%', '  test error: 89.68%')
Total error: 0.047139885027
('epoch:   13', '  train error: 89.56%', '  test error: 89.72%')
Total error: 0.0471459617826
('epoch:   14', '  train error: 90.13%', '  test error: 90.20%')
Total error: 0.0471302472029
('epoch:   15', '  train error: 90.07%', '  test error: 89.68%')
Total error: 0.0471525497087
('epoch:   16', '  train error: 90.26%', '  test error: 90.18%')
Total error: 0.0471419186775
('epoch:   17', '  train error: 90.97%', '  test error: 91.08%')
Total error: 0.0471563384617
('epoch:   18', '  train error: 88.76%', '  test error: 88.65%')
Total error: 0.0471480887109
('epoch:   19', '  train error: 90.97%', '  test error: 91.08%')
Total error: 0.0471466129654
('epoch:   20', '  train error: 90.08%', '  test error: 89.91%')
Total error: 0.0471322920471
('epoch:   21', '  train error: 90.08%', '  test error: 89.91%')
Total error: 0.0471505302295
('epoch:   22', '  train error: 88.76%', '  test error: 88.65%')
Total error: 0.0471490334095
('epoch:   23', '  train error: 90.97%', '  test error: 91.08%')
Total error: 0.0471643256975
('epoch:   24', '  train error: 90.25%', '  test error: 90.26%')
Total error: 0.0471597744749
('epoch:   25', '  train error: 88.76%', '  test error: 88.65%')

This took several hours :-/

Classification with PyBrain

info@martin-thoma.de (Martin Thoma) — Mon, 16 Jun 2014 01:02:00 +0200

PyBrain is a Python library for machine learning. It’s in version 0.31 and the last change is 2 months ago (source). The source code is licensed under BSD 3 Clause License. The documentation is usable, but for from perfect.

Classification example

The following is a slightly modified example from the documentation. It shows how PyBrain starts learning to classify 2-dimensional datapoints into 3 classes:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

# Source: http://pybrain.org/docs/tutorial/fnn.html
from pybrain.datasets import ClassificationDataSet
from pybrain.utilities import percentError
from pybrain.tools.shortcuts import buildNetwork
from pybrain.supervised.trainers import BackpropTrainer
from pybrain.structure.modules import SoftmaxLayer

# Only needed for data generation and graphical output
from pylab import ion, ioff, figure, draw, contourf, clf, show, plot
from scipy import diag, arange, meshgrid, where
from numpy.random import multivariate_normal
from random import normalvariate


def generate_data(n=400):
    INPUT_FEATURES = 2
    CLASSES = 3
    means = [(-1, 0), (2, 4), (3, 1)]
    cov = [diag([1, 1]), diag([0.5, 1.2]), diag([1.5, 0.7])]
    alldata = ClassificationDataSet(INPUT_FEATURES, 1, nb_classes=CLASSES)
    minX, maxX = means[0][0], means[0][0]
    minY, maxY = means[0][1], means[0][1]
    for i in range(n):
        for klass in range(CLASSES):
            features = multivariate_normal(means[klass], cov[klass])
            x, y = features
            minX, maxX = min(minX, x), max(maxX, x)
            minY, maxY = min(minY, y), max(maxY, y)
            alldata.addSample(features, [klass])
    return {'minX': minX, 'maxX': maxX,
            'minY': minY, 'maxY': maxY, 'd': alldata}


def generate_data2(n=400):
    alldata = ClassificationDataSet(2, 1, nb_classes=2)
    minX, maxX = 3, 3
    minY, maxY = 2, 2
    for i in range(1000):
        x = normalvariate(3, 0.6)
        y = normalvariate(2, 1)
        minX, maxX = min(minX, x), max(maxX, x)
        minY, maxY = min(minY, y), max(maxY, y)
        alldata.addSample((x, y), (0,))
    for i in range(1000):
        x = normalvariate(7, 0.5)
        y = normalvariate(1, 0.1)
        alldata.addSample((x, y), (1,))
    return {'minX': minX, 'maxX': maxX,
            'minY': minY, 'maxY': maxY, 'd': alldata}


def perceptron(hidden_neurons=5, weightdecay=0.01, momentum=0.1):
    INPUT_FEATURES = 2
    CLASSES = 3
    HIDDEN_NEURONS = hidden_neurons
    WEIGHTDECAY = weightdecay
    MOMENTUM = momentum

    # Generate the labeled set
    g = generate_data()
    #g = generate_data2()
    alldata = g['d']
    minX, maxX, minY, maxY = g['minX'], g['maxX'], g['minY'], g['maxY']

    # Split data into test and training dataset
    tstdata, trndata = alldata.splitWithProportion(0.25)

    trndata._convertToOneOfMany()  # This is necessary, but I don't know why
    tstdata._convertToOneOfMany()  # http://stackoverflow.com/q/8154674/562769

    print("Number of training patterns: %i" % len(trndata))
    print("Input and output dimensions: %i, %i" % (trndata.indim,
                                                   trndata.outdim))
    print("Hidden neurons: %i" % HIDDEN_NEURONS)
    print("First sample (input, target, class):")
    print(trndata['input'][0], trndata['target'][0], trndata['class'][0])

    fnn = buildNetwork(trndata.indim, HIDDEN_NEURONS, trndata.outdim,
                       outclass=SoftmaxLayer)

    trainer = BackpropTrainer(fnn, dataset=trndata, momentum=MOMENTUM,
                              verbose=True, weightdecay=WEIGHTDECAY)

    # Visualization
    ticksX = arange(minX-1, maxX+1, 0.2)
    ticksY = arange(minY-1, maxY+1, 0.2)
    X, Y = meshgrid(ticksX, ticksY)

    # need column vectors in dataset, not arrays
    griddata = ClassificationDataSet(INPUT_FEATURES, 1, nb_classes=CLASSES)
    for i in range(X.size):
        griddata.addSample([X.ravel()[i], Y.ravel()[i]], [0])

    for i in range(20):
        trainer.trainEpochs(1)
        trnresult = percentError(trainer.testOnClassData(),
                                 trndata['class'])
        tstresult = percentError(trainer.testOnClassData(
                                 dataset=tstdata), tstdata['class'])

        print("epoch: %4d" % trainer.totalepochs,
              "  train error: %5.2f%%" % trnresult,
              "  test error: %5.2f%%" % tstresult)
        out = fnn.activateOnDataset(griddata)
        # the highest output activation gives the class
        out = out.argmax(axis=1)
        out = out.reshape(X.shape)

        figure(1)  # always print on the same canvas
        ioff()  # interactive graphics off
        clf()   # clear the plot
        for c in [0, 1, 2]:
            here, _ = where(tstdata['class'] == c)
            plot(tstdata['input'][here, 0], tstdata['input'][here, 1], 'o')
        if out.max() != out.min():  # safety check against flat field
            contourf(X, Y, out)   # plot the contour
        ion()  # interactive graphics on
        draw()  # update the plot

    ioff()
    show()

if __name__ == '__main__':
    from argparse import ArgumentParser, ArgumentDefaultsHelpFormatter

    parser = ArgumentParser(formatter_class=ArgumentDefaultsHelpFormatter)

    # Add more options if you like
    parser.add_argument("-H", metavar="H", type=int, dest="hidden_neurons",
                        default=5,
                        help="number of neurons in the hidden layer")
    parser.add_argument("-d", metavar="W", type=float, dest="weightdecay",
                        default=0.01,
                        help="weightdecay")
    parser.add_argument("-m", metavar="M", type=float, dest="momentum",
                        default=0.1,
                        help="momentum")
    args = parser.parse_args()

    perceptron(args.hidden_neurons, args.weightdecay, args.momentum)

See it in action:

Resources

Official website
StackExchange
- StackOverflow: 134 questions
- stats.SE
reddit.com/r/MachineLearning

Reference Management with JabRef

info@martin-thoma.de (Martin Thoma) — Sun, 15 Jun 2014 20:51:00 +0200

Keeping track of papers, articles and books or more general sources you can cite is a task you will have to tackle when you’re writing your thesis. One way to store information is via BibTeX files.

BibTeX is a reference management software that is used together with LaTeX. If you’re new to BibTeX or references in LaTeX in general you could read the followin articles:

bibtex vs. biber and biblatex vs. natbib
What is the difference between bibtex and biblatex? - I keep forgetting that all the time.

I’ve tried to create a MWE for biblatex, but as I don’t really know much about BibTeX it’s probably not the best resource to start with.

A longer working example of BibTeX is my bachelors thesis (work is still in (slow) progress).

What is JabRef?

JabRef is a reference management software that uses BibTeX as its native format. It looks like this:

JabRef

How can I get it?

JabRef 2.10 beta is part of the Debian sources. Thus it can be installed by

sudo apt-get install jabref

The official development page is jabref.sourceforge.net. As it is written in Java it can be installed on all (or at least most) systems where you have a JVM (thus: good news Windows / Mac users!).

What’s good about JabRef?

You can easily add the information where the PDF is located on your system and open the PDF directly via JabRef:

Adding PDF with JabRef

This is especially powerfull with the search. As you can add quite a lot of information to the entries (such as the abstract and comments) and searching that is faster / easier than searching folders of PDFs, it’s very convenient to use JabRef for opening your PDFs:

Searching with JabRef

The preview also helps you to see how it might appear in your document:

JabRef preview

Another nice feature is the autocompletion of authors names and titles.

What could be better?

It would be neat if you could automatically upload your BibTeX file and use other peoples BibTeX files to complete the information or to get informed of possible errors in your database.

Additionally, a good PDF reader that is fast and allows annotations of which JabRef would be aware would be awesome.

Speed is quite often an issue with Java applications in my experience. It takes about 5 seconds to open my BibTeX file of about 37 references. After starting that it’s ok.

Shortcuts are important for tools that get used often, too. Although JabRef has some reasonable shortcuts like Ctr+i for importing a BibTeX file and Ctr+s for saving the current BibTeX file, I miss Ctrl+f for searching the database.

Importing other BibTeX files into the current BibTeX file is possible. But I would also like to “import” BibTeX data by copy and pasting a single dataset.

Alternatives

Mendeley is a closed-source alternative with a built-in PDF reader. It seems to be fast and can complete information when the title is given.

Discussions with friends

info@martin-thoma.de (Martin Thoma) — Sat, 10 May 2014 02:04:00 +0200

I’ve recently had a very interesting discussion with some friends. We talked about a lot of stuff and I don’t want to go into detail about the topics we’ve discussed, but rather some thoughts about discussions.

Types of discussions

There are many ways people talk to each other:

Small Talk is about being together and feeling fine. We do it when we are together waiting for something or just because it feels good. It’s not so much about content which show discussions about weather, fashion or sitcoms, although the amount of information you exchange varies quite a lot.
Feelings and talking about feelings is difficult, but important. Although I tend to say it’s like small talk, but I don’t want to go into detail here.
Teaching is a situation where one person knows more than another person in some topic he (or she) teaches the other.
Learning is either the other side in the teaching situation or learning together. I think it is fundamentally different and deserves an own bullet point because in the teaching situation, the teacher leads the conversation. In the learning situation you don’t have somebody who leads the conversation. It’s a free flow of (hopefully) constructive thoughts with the aim to get a new ability as a group.
Moral or Political Discussions are some sort of learning, but in contrast to learning in the sense that you get another ability or get to know some more facts, the aim is on evaluating how to weight those facts. Discussions involve a learning part, but the core is about putting things together. So it is not mainly about what could be done but what should be done and for what reasons.

What’s special about discussions?

Discussions involve some aim. That aim might be

What is the best way to explain XYZ? What does best mean in this context?
What should we do in situation ABC? What is to consider for what reasons?

But very much of the conversations that are labeled as “discussions” feel rather like small talk for the following reasons:

I think to have a meaningful discussion you need to have a partner who is not of your opinion. For political discussions this means the following are NOT meaningful statements:

I want everybody to be able to find a job (or shorter: Fight unemployment)
I want to make less dept
I want everybody to be able to live a healthy live
I want to help families
I want everybody to be happy

All these three statements are statements that everybody could agree with. But you get reasonable statements when you tell how to do so.

You can also talk about the “order”: I want to make less dept, but when young families have to pay more than 20 Euro per year more it is not acceptable. Of course, that’s still much to vague, but I think you get the point. Telling what you value might help people to intuitively decide with which person they could rather identify, but it is not enough to name reasons.

Another important part of a meaningful discussion is that it should in theory be possible to change the mind of the other person. This is the reason why I’ve called it “discussions with friends”. In contrast to the truth-finding “discussions” you have in science, you don’t need to be able to say what observations could falsify your theory. But if you go into a discussion and you know that it is impossible that the other - no matter how good he makes his point - convinces you of his opinion, you’re actually not having a discussion. You try to teach them. (And it’s a bad type of teaching.)

Techniques to take most out of discussions

It is difficult to be open minded in a good discussion. You might need to end a discussion and think through it some time after you had it. Because we tend to defend a point of view when we begin to defend it, no matter how good the other makes his point. It is very difficult to honestly say: Yes, you’re correct. That changes the case and probably my opinion.
And you should not say so too fast. Some people are charismatic, some can speak or write very good. When you hear a new argument the first time, you might have difficulties to find a correct counter argument. But when you’re with friends you should be able to say:

That’s the first time I’ve heard that. I have to think about it.

or something like

I’ve never seen it from that perspective.

People like to teach and this makes them feel more comfortable discussing with you. You don’t give up your opinion with statements like these, but it’s simply being honest.

Speaking of honesty, there is a similar DON’T: Don’t say something like

Yes, I agree with you, but [Whatever he just said negated]

I hear this so often in political debates by amateurs who have read something about discussions. By agreeing you make it easier for your discussion partner to agree with you (he can’t disagree when you agree with him, can he?), but it should be a honest way to agree. It’s a very bad step to fake agreement.

Group dynamics

Always - and that’s not only related to discussions - try to think what and how the other thinks. Sometimes when a group discusses, a single person get’s isolated and has to defend his point of view alone. That’s ok. But don’t make it worse for him:

Give him enough time to think and to talk.
Only one person and argument at a time. It’s difficult to switch and to concentrate on two (or even more) separate discussions.
Try to support him when he has trouble with finding the correct words or the best way to express himself. It is difficult to tell when you simply have to wait and give him some time to think and when you should try to support him.

Know when you’re wrong

Changing opinions is difficult. Especially if you have them for a long time or when you’ve already defended them. It’s a little bit like an investment. When you’ve already spend 6,000 Euro for an investment and there is a tiny chance that you will get what you want but a growing chance that the investment was just a bad idea and probability suggest to change the investment and rescue what’s left, most people will not do so. Somehow we quite often tend to a all-or-nothing thinking which is suboptimal in many cases.

So, what can we do so to figure that out for ourselves? Sometimes it is impossible to do so while discussing, but after the discussion you can think about it:

What arguments did I tell the others? How did I tell them? What were counter arguments? Are they valid? If so / not: Why?
Are there arguments I’ve missed?
Do I feel comfortable with my arguments? Why do I think they could be weak? What do I think are strong arguments? Do others also think that those arguments are strong?

If you’re not sure about it, you can continue it. It should not be a simple repetition of arguments, but a new discussion with a new structure of arguments. If you’re still not sure after many arguments, you can make a pro / contra list.

Please note:

It’s not important what the majority thinks. However, it might be an indicator.
Some arguments might simply be wrong. But don’t make this choice too easily.
Some arguments might be irrelevant. I tend to think “How does this impact my life?”. If an argument has no impact or simply assumes an unrealistic situation, it might be irrelevant.
You don’t have to agree at the end. A honest disagreement where both sides understand the arguments of the other side is much better than a comprise that no side really thinks is good.

And I want to note that discussions might have many equally valid answers, whereas science has only one correct answer (which might be difficult to find / almost impossible to decide which thesis is correct, but there is only one).

HTML5 Template

info@martin-thoma.de (Martin Thoma) — Sat, 03 May 2014 20:43:00 +0200

Once in a while I need to create simple HTML pages. This is the template I use:

<!doctype html>
<html lang="en">
<head>
    <meta charset="utf-8" />
    <title>TODO</title>
</head>
<body>
    <h1>TODO</h1>
</body>
</html>

and as a sublime snippet:

<snippet>
    <content><![CDATA[
<!doctype html>
<html lang="en">
<head>
    <meta charset="utf-8" />
    <title>${1:Title}</title>
</head>
<body>
    <h1>${2:Header}</h1>
    ${3:Content}
</body>
</html>
]]></content>
    <tabTrigger>html5</tabTrigger>
    <!-- Optional: Set a scope to limit where the snippet will trigger -->
    <!-- <scope>source.python</scope> -->
</snippet>

Prüfungsverwaltung am KIT

info@martin-thoma.de (Martin Thoma) — Tue, 08 Apr 2014 16:09:00 +0200

Gerade ist mir aufgefallen, dass ich nicht zur Prüfung “Programmierparadigmen” angemeldet bin. Dabei war ich mir relativ sicher, mich sogar am ersten Tag angemeldet zu haben, als die Anmeldung freigeschaltet wurde.

Meiner Meinung nach sind wir Studenten hier in einer ungerechtfertigt schlechten Position: Sollte tatsächlich ein Fehler passiert sein und meine Anmeldung nicht funktioniert haben bzw. die Anmeldung (wie oder warum auch immer) rückgängig gemacht worden sein, habe ich keinerlei Möglichkeit zu beweisen, dass ich angemeldet war und nicht etwa es einfach vergessen habe. Was ich in meinem konkreten Fall nicht ausschließen will, schließlich ist das schon zwei Monate her und ich habe mich für einige Prüfungen angemeldet und keine automatische Bestätigung der Anmeldung erhalten.

Verbesserungsvorschlag

Kurz und gut: Ich will, dass die Uni ein digitales Signaturverfahren einsetzt und die technischen Möglichkeiten nutzt, um den Studenten zu helfen ihre Rechte durchzusetzen. Wie das genau funktionieren soll, wird im Folgenden erklärt.

Ausführlich

Zur Verbesserung dieser Situation schlage ich folgendes vor:

Studenten sollen die Möglichkeit bekommen, einen öffentlichen Schlüssel hochzuladen:

Einen Schlüssel hinzufügen

Mit diesem signieren sie Prüfungsanmeldungen:

Prüfungsanmeldung signieren

Bei jeder Prüfungsanmeldung soll innerhalb von 24h eine E-Mail in Textform (keine PDF) an u**@student.kit.edu, also die KIT E-Mail-Adresse des Studenten, geschickt werden, der sich angemeldet hat.

Diese E-Mail soll folgendes beinhalten:

Alle Prüfungen (mit Name, Termin der Prüfung, letzter An- und Abmeldetermin), zu den der Student angemeldet ist.
Der volle Name und die Matrikelnummer des Studenten.
Das Datum der E-Mail.

Diese E-Mail soll mit einem offiziellen KIT Schlüssel signiert werden. Dies könnte z.B. mit PGP gemacht werden und würde dann etwa so aussehen:

—–BEGIN PGP SIGNED MESSAGE—–
Hash: SHA1

Student: Martin Thoma (Matrikelnummer: 1612345)
Zeitpunkt: 08.04.2014, 12:34:56 Uhr
Angemeldete Prüfungen:
* Programmierparadigmen (Prüfungstermin: 10.04.2014; letzter Termin der Anmeldung: 31.03.2014; letzer Termin der Abmeldung: 08.04.2014)
* Kognitive Systeme (Prüfungstermin: 11.04.2014; letzter Termin der Anmeldung: 15.03.2014; letzer Termin der Abmeldung: 10.04.2014)

—–BEGIN PGP SIGNATURE—–
Version: GnuPG v1.4.14 (GNU/Linux)

iEYEARECAAYFAlNDzBoACgkQO3Q6GUuCW82z+ACfamkVC/S8HIpASH8F0ZGVbVW1
rgwAn2cRvNeDN3pZVTpvNWV1vYK9f1fI
=3DvZU7
—–END PGP SIGNATURE—–

Der Student sollte jederzeit die Möglichkeit haben, eine wie oben beschriebene signierte E-Mail mit der Liste der Prüfungen und einem Datum zu erhalten.

Nun sollen Studenten in der Klausur die Möglichkeit haben, diese E-Mail auf einem USB-Stick mitzubringen. Die Aufsicht müsste also einen Computer haben, mit dem sie die E-Mail anschauen und insbesondere die Signatur überprüfen können. So könnte ein Student, der einmal zu einer Klausur angemeldet das auch belegen.

Abmeldungen

Die auf den ersten Blick sehr schöne Lösung hat einen wichtigen Schönheitsfehler: Studenten können sich von Prüfungen abmelden. Daher ist es möglich, dass ein Student eine signierte E-Mail hat, die ihm die Anmeldung zur Prüfung bestätigt, er aber dennoch nicht zur Prüfung angemeldet ist, da er sich abgemeldet hat.

Deshalb wäre es nötig, dass jeder Student ein eigenes Schlüsselpaar aus einem öffentlichen und einem privaten Schlüssel erstellt. Wenn sich ein Student von der Prüfung abmeldet, bekommt er einen Text, z. B. etwas in dieser Richtung:

Ich, Martin Thoma (Matrikelnummer: 1612345), melde mich hiermit am 08.04.2014 um 12:34:56 Uhr von der Prüfung “Programmierparadigmen” (Prüfungsdatum: 10.04.2014) ab.

Diese muss er mit seinem privaten Schlüssel signieren. Die Uni muss daher den öffentlichen Schlüssel des Studenten kennen und diesem zugeordnet haben. Sobald die Abmeldung eingegangen ist, wird innerhalb von 24h eine Bestätigung an den Studenten verschickt, die von der oben beschriebenen Form ist.

oder auch eine Fehlermitteilung:

Sehr geehrter Herr Thoma,
die Signatur ihrer Nachricht war ungültig. Bitte überprüfen Sie, ob Sie den korrekten privaten Schlüssel verwendet haben.

Nun hätte auch die Klausuraufsicht die Möglichkeit zu belegen, dass ein Student sich von der Prüfung abgemeldet hat.

Technik-Afinität

Was ist mit Studenten, die es nicht schaffen ein Schlüsselpaar zu erzeugen bzw. eine Nachricht zu signieren?

Nun, das ist einfach: Wer keinen öffentlichen Schlüssel hinterlegt, bekommt auch keine signierten Nachrichten. Dennoch sollten diese Studenten die E-Mails, wie sie oben beschrieben wurden, bekommen. Dann weiß man als Student, dass vermutlich alles mit der Anmeldung geklappt hat bzw. wenn die E-Mail nicht kommt, dass etwas nicht geklappt hat.

Bevor also die erste signierte E-Mail der Uni an den Studenten geschrieben wird, muss der Student seinen öffentlichen Schlüssel der Uni mitgeteilt haben und eine Nachricht mit dem privaten Schlüssel signiert haben. Damit wird sichergestellt, dass der Student prinzipiell in der Lage ist sich von Prüfungen abzumelden.

Student verliert Schlüssel

Was macht man, wenn ein Student einmal einen Schlüssel eingericht hat, diesen aber verliert?

Was passiert, wenn man seinen Studentenausweis verliert? Ich denke die Vorgehensweise wäre dann ähnlich.

Vorstellbar wäre etwas in der Art:

Der Student muss zum Studienbüro und Schriftlich bestätigen, dass der öffentliche Schlüssel aus dem KIT-System entfernt wird und damit ungültig wird.
Der Student muss sich bei allen Professoren persönlich melden und unterschreiben, wenn er sich von der Klausur abmelden will.

PGP - Was ist das?

Siehe Wikpedia.

Es ist für folgende Systeme verfügbar:

Linux, siehe z.B. Ubuntu mit GnuPG
Windows, siehe z.B. GPG4Win
Mac, siehe z.B. Mac GNU Privacy Guard

Eine weitere Erklärung ist im Piratenwiki.

Einfügen ins System

Die Website wechall.net bietet die Möglichkeit, einen PGP-Schlüssel hochzuladen. Sobald man das gemacht hat, werden dort alle Nachrichten mit dem öffentlichen Schlüssel verschlüsselt. Der Quellcode der Seite ist hier verfügbar.

Mit PHP scheint das ganze sehr einfach zu sein (Quelle). Auch mit Python sieht die Sache sehr leicht aus (Quelle).

Da ich keine Ahnung habe was für campus.kit.edu verwendet wird, kann ich hier leider nicht mehr dazu sagen.

Dann würde man noch eine Tabelle in der Datenbank benötigen. Die würde etwa so aussehen:

CREATE TABLE IF NOT EXISTS "openpgp_keys" (
  "id" int(11) NOT NULL AUTO_INCREMENT,
  "student_id" int(11) NOT NULL,
  "public_pgp_key" text NOT NULL,
  "upload_date" datetime NOT NULL,
  "confirmation_date" datetime DEFAULT NULL,
  "is_activated" tinyint(1) NOT NULL DEFAULT '0',
  PRIMARY KEY ("id")
) AUTO_INCREMENT=1;

oder graphisch:

openpgp_keys Tabelle

Ressourcen

Sollte jemand den Vorschlag modifizieren wollen, hier ist das HTML dazu:

<div class="content_full_portal">
        <h1>PGP/GPG</h1>
        <h2>Neuen Schlüssel hinzufügen</h2>
        <div style="width: 100%;">
            <div class="gwf3_formY">
                <form action="/account" method="post" enctype="multipart/form-data">
                    <table>
                        <tbody>
                            <tr><td>Öffentlichen Schlüssel als Datei hochladen</td><td>
                            </td><td><input type="file" name="gpg_file"></td></tr>
                            <tr><td colspan="3">oder hier einfügen</td></tr>
                            <tr><td colspan="3"><textarea id="gpg_paste" name="gpg_paste" cols="80" rows="8"></textarea></td></tr>
                            <tr><td></td><td></td><td><input type="submit" name="setup_gpg" value="Schlüssel hochladen"></td></tr>
                            <tr><td colspan="3"><input type="hidden" name="gwf3_csrf" value="gp1yqWh4"></td></tr>
                        </tbody>
                    </table>
                </form>
            </div>
        </div>

        <h2>Liste bekannter Schlüssel</h2>
        <table style="width:100%">
            <tbody><tr>
                <th>Name des Schlüssels</th>
                <th>Datum des Uploads</th>
                <th>Bestätigt</th>
                <th>Aktiviert</th>
            </tr>
            <tr>
                <td>sdfasdf asdfasd asdf</td>
                <td>12.10.2013, 14:45 Uhr</td>
                <td>am 12.10.2014, 16:00 Uhr</td>
                <td>Ja (<a href="">Deaktivieren</a>)</td>
            </tr>
            <tr>
                <td>asdfasdf asdf asdfa sdf &nbsp;</td>
                <td>02.04.2014, 06:01 Uhr</td>
                <td><a href="">noch nicht</a></td>
                <td>Nein (<a href="">Aktivieren</a>)</td>
            </tr>
        </tbody></table>

        
    </div>

und

<div class="content_full_portal">
    <h1>Prüfungsanmeldung</h1>
    Signieren Sie folgende Nachricht mit ihrem Schlüssel:<br>

    <a href="">Nachricht als Textdatei herunterladen</a><br>

    oder<br>

    Nachricht zum kopieren:<br>
    <textarea style="width: 800px;height: 60px;">Hiermit melde ich, Martin Thoma (Matrikelnummer: 1612345), mich heute (28.03.2014, 12:34:56 Uhr) zur Prüfung 'Programmierparadigmen', die am 10.04.2014 statt findet, an. Mir ist bekannt, dass der letzte Zeitpunkt der Abmeldung am 08.04.2014 ist.</textarea>
    <h2>Anmeldung durchführen</h2>
    <form>
        <label for="filet">Signierte Bestätigung als Textdatei hochladen:</label><br>
        <input type="file" id="filet"><br>

        <p>oder signierte Bestätigung direkt einfügen:<br></p>
        <label for="textareat"></label>
        <textarea id="textareat" style="width: 800px;height: 60px;"></textarea><br>
        <input type="submit">
    </form>
</div>

Weitere Probleme der Prüfungsorganisation

In diesem Artikel will ich nur die Prüfungsverwaltung diskutieren. Dennoch sollte darauf hingewiesen werden, dass Weiteres nicht gerade optimal gelöst ist:

Informationspolitik: Bereits zu Semesterbegin sollte folgendes bekannt sein:
- Zeitpunkt der Prüfung
- Letztmöglicher Zeitpunkt der Anmeldung
- Letztmöglicher Zeitpunkt der Abmeldung
Die Anmeldungsfreischaltung der Prüfungen finden zu sehr unterschiedlichen Zeitpunkten statt. Die Anmeldung sollte breits zu Semesterbegin für alle Klausuren möglich sein.
Die Einsicht ist immer schlecht organisiert. Da sich die Prüfungstermine größtenteils in der vorlesungsfreien Zeit befinden und darin aber stark gestreut sind (manche sind zu Beginn, manche in der Mitte, manche am Ende) sind die Zeitpunkte, zu denen man als Student Ferien hat sehr stark eingeschränkt. Wenn man dann erst nach der Klausur erfährt, wann die Einsicht sein wird, kann man Ferien komplett vergessen, weil man in Karlsruhe bleiben muss um auf den Termin der Einsicht zu warten.
Bei den Physikern ist wenigstens die Korrektur immer sehr schnell, sodass man davon ausgehen kann, dass die Einsicht wenige Tage nach der Prüfung statt findet. Bei den Informatikern … naja, da hat man ja noch Glück wenn sie im selben Monat ist. Und man es dann rechtzeitig erfährt.
Lösungen und Notengrenzen sind nicht in jeder einsicht vorhanden bzw. klar. Gerade wenn nicht klar ist, mit welcher Punktzahl man welche Note bekommt könnte ein Fehler passieren, den man nicht überprüfen kann. Nach diesen Informationen sollte man nicht in der Einsicht fragen müssen. Sie sollten in der Einsicht direkt verfügbar sein.

Schlusswort

Welche Vorteile hat das beschriebene Verfahren gegenüber der momentanen Situation?

Studenten können beweisen, dass sie zur Prüfung angemeldet sind / nicht sind
Prüfer können beweise, dass Studenten angemeldet sind / nicht sind

Selbst wenn man den Teil mit der asymmetrischen Verschlüsselung nicht macht, hätte man als Student zumindest ein bisschen was in der Hand und einen Mechanismus, der automatisch Feedback gibt, ob alles geklappt hat. Im Gegensatz zu der momentanen Situation, wo wir absolut nichts belegen können und man sehr leicht übersehen kann, wenn etwas bei der Anmeldung schief gegangen ist.

Was haltet ihr davon? Hattet ihr auch schon solche Probleme mit der Prüfungsanmeldung?

Prolog

info@martin-thoma.de (Martin Thoma) — Wed, 02 Apr 2014 16:36:00 +0200

Prolog is a logic programming language.

Any prolog program consists of

declaration of facts (also called objects)
rules about objects and their relationship
asking “questions” about objects and their relationship

First steps

At first you have to install swi-prolog. After installing it, you can call it with swipl.

When you’ve started swipl you can import your myscript.pl files by consult(myscript).. Don’t forget the point at the end. It is important and somehow like the ; in C.

Defining facts

The definition of facts can look like this:

loves(john,X).

Note that names of all relationships and all objects have to begin with a lower case letter. Variables begin with a capital letter.
The relationship is written first (john). The objects are separated by , and everything is finished with ..
The example from above means: john loves everything.
You can have any number of arguments.
The ordering of arguments is important (loves(john,X) is not the same as loves(X,john)).

Symbols

Boolean	Prolog
and	`,`
or	`;`
if	`:-`
not	`not`

An underscore _ is an anonymous variable.

Rules

Rules consist of a head and a body. They are separated by :-.

Questions

?own(mary,book)

This will invoke the prolog unifier. So Prolog will check if the fact own(mary,book) can be derived from the known facts. Important is that it starts with ?. Everything else in questions is just like it was for facts.

Fox, goose and bag of beans puzzle

This puzzle helped me a lot to understand how Prolog works. It goes as follows:

Once upon a time a farmer went to market and purchased a fox, a goose, and a bag of beans. On his way home, the farmer came to the bank of a river and rented a boat. But in crossing the river by boat, the farmer could carry only himself and a single one of his purchases - the fox, the goose, or the bag of the beans. If left together, the fox would eat the goose, or the goose would eat the beans. The farmer’s challenge was to carry himself and his purchases to the far bank of the river, leaving each purchase intact. How did he do it?

Source: Wikpedia

Let’s call the side of the river where the farmer currently is ‘left’ and the other side ‘right’ to make things easier.

When you solve this by hand, you should note the following:

The only two things that the farmer can leave alone is the fox and the bag of beans.
As the farmer is always in the boat when anything else is in the boat, you can ignore that step. Just think of the farmer and possibly one other thing teleporting from one side to the other.
You only have to look at what’s left and what’s right. If nothing gets eaten on the left side and nothing gets eaten on the right side, you’ve got a valid situation.
Whenever a situation occurs twice, you did some redundant steps. In other words: At least one solution exists, where no repetitions take place. The shortest solution is one of them.

To solve this task, you split the problem into parts:

Define a rule valid that takes a 4-tuple, where the first element is the position of the man, the second of the goose, then the fox and then the beans. Positions can either be left or right.
Define a rule step(S1, Description, S2) that makes one step from situation S1 to situation S2 where Description contains the information what was brought to the other side. The situations are, as above, 4-tuples.
Define a rule reachable(S1, SituationList, Steps, S2) that is true if S2 can be reached by any number of steps from S1. The list of steps done to get from S1 to S2 is stored in Steps. The SituationList is used to prevent doing unnecessary redundant steps.

So lets get through this.

There are essentially three situations that can occur:

The man is on the same side as the goose,
the man is on the same side as the beans or
the man is on the same side as the fox.

If the man is on the same side as the goose, everything is fine. Nobody gets eaten / eats when the man is there.

If the man is where the beans are, then the goose should not be where the fox is. The case that the goose is where the man is was done before.

And the last case, the man is where the fox is. Now we have to make sure that the beans are not where the goose is:

p(left).
p(right).
valid((Man, Goose, Fox, Beans)) :- p(Man),p(Goose),p(Fox),p(Beans), Man == Goose.
valid((Man, Goose, Fox, Beans)) :- p(Man),p(Goose),p(Fox),p(Beans), Man == Beans, Goose \= Fox.
valid((Man, Goose, Fox, Beans)) :- p(Man),p(Goose),p(Fox),p(Beans), Man == Fox, Goose \= Beans.

The step is also quite simple. There are four possible actions you could take at each situation:

Only the man goes to the other side,
the man and the beans / goose / fox go the the other side.

So here is the code:

step((Man1, Goose, Fox, Beans), 'Man', (Man2, Goose, Fox, Beans)) :- 
    Man1 \= Man2.

step((Man1, Goose1, Fox, Beans), 'Goose', (Man2, Goose2, Fox, Beans)) :- 
    Man1 \= Man2, Goose1 \= Goose2.

step((Man1, Goose, Fox1, Beans), 'Fox', (Man2, Goose, Fox2, Beans)) :- 
    Man1 \= Man2, Fox1 \= Fox2.

step((Man1, Goose, Fox, Beans1), 'Beans', (Man2, Goose, Fox, Beans2)) :- 
    Man1 \= Man2, Beans1 \= Beans2.

And finally the reachable part.

Either, you’re already at the situation where you want to get to. So you don’t have to do any step and it doesn’t matter which situations you have seen so far.
Or you want to get from S to G. That is only possible, when you can get from S to an intermediate situation with one step less:

reachable(S, _, [], S).
reachable(S, Visited, [Step|Steps], G) :- 
    step(S, Step, Tmp), valid(Tmp), not(member(Tmp, Visited)),
    reachable(Tmp, [Tmp|Visited], Steps, G).

Putting it together:

p(left).
p(right).
valid((Man, Goose, Fox, Beans)) :- p(Man),p(Goose),p(Fox),p(Beans), Man == Goose.
valid((Man, Goose, Fox, Beans)) :- p(Man),p(Goose),p(Fox),p(Beans), Man == Beans, Goose \= Fox.
valid((Man, Goose, Fox, Beans)) :- p(Man),p(Goose),p(Fox),p(Beans), Man == Fox, Goose \= Beans.

step((Man1, Goose, Fox, Beans), 'Man', (Man2, Goose, Fox, Beans)) :- 
    valid((Man2, Goose, Fox, Beans)),
    Man1 \= Man2.

step((Man1, Goose1, Fox, Beans), 'Goose', (Man2, Goose2, Fox, Beans)) :- 
    valid((Man2, Goose2, Fox, Beans)),
    Man1 \= Man2, Goose1 \= Goose2.

step((Man1, Goose, Fox1, Beans), 'Fox', (Man2, Goose, Fox2, Beans)) :- 
    valid((Man2, Goose, Fox2, Beans)),
    Man1 \= Man2, Fox1 \= Fox2.

step((Man1, Goose, Fox, Beans1), 'Beans', (Man2, Goose, Fox, Beans2)) :- 
    valid((Man2, Goose, Fox, Beans2)),
    Man1 \= Man2, Beans1 \= Beans2.

reachable(S, _,[], S).
reachable(S, Visited, [Step|Steps], G) :- 
    step(S, Step, Tmp), not(member(Tmp, Visited)),
    reachable(Tmp, [Tmp|Visited], Steps, G).

start((left,left,left,left)).
goal((right,right,right,right)).

solve(Steps) :- start(S), goal(G), reachable(S, [], Steps, G).

You can load this with swipl -s puzzle.pl and get all solutions with bagof(X, solve(T), L). When you press ; after the first solution, you get the second solution. (Yes, this riddle has only two solutions that avoid redundant steps.)

Material

Introduction to PROLOG: One hour of introduction to prolog by Shashi Bhushan. Much of what I have written here is based on this video.
SWI Prolog: A great resource for looking things up.
Facts, Rules and Queries
Simple Prolog example

TCL

info@martin-thoma.de (Martin Thoma) — Wed, 02 Apr 2014 04:04:00 +0200

TCL is an imperative, interpreted, dynamically typed programming language. The Tool Command Language appeared in 1988. In Tcl, everything is a string.

It is probably best (and completely?) described by the 12 Rules for Tcl.

Basic Syntax by Example

Some nice comparisons of the syntax of many languages can be found at rigaux.org

Hello World

set myVariable "Hallo World!"
puts $myVariable

Loop over a list

% foreach element {1 2 3} {
    puts $element
}

Fibonacci

proc fib {n} {
    if {$n < 2} {
        return $n
    } else {
        return [expr {[fib [expr {$n-2}]] + [fib [expr {$n-1}]]}]
    }
}

for {set x 0} {$x<10} {incr x} {
    puts [fib $x]
}

Comments

TCL uses # as a line comment and does not offer block comments. In fact, they recommend this:

if 0 {
    Any Tcl code to be commented out (with matching braces, of course!)
    or any other kind of text, will be ignored - and even [exit] won't fire
    because it's in braces, so left unevaluated!
}

Source: wiki.tcl.tk/1669

Logical operators

AND	OR	TRUE	FALSE
&&	\|\|	1	0
EQUAL	UNEQUAL	NOT
==	!=	!

99 bottles of beer

#!/usr/bin/tclsh

# 99.tcl; Tcl version of 99 Bottles of Beer Song

proc findBString { count } {
  return [switch -exact -- $count {
    0       { expr {"No more bottles"}  }
    1       { expr {"1 bottle"}         }
    default { expr {"$count bottles"}   }
  }]
}

set bottles 99
set bString [findBString $bottles]
while {$bottles + 1} {
  puts "$bString of beer on the wall.  $bString of beer."
  incr bottles -1
  if {$bottles + 1} {
    set bString [findBString $bottles]
    puts "Take one down, pass it round, $bString of beer on the wall.\n"
  } else {
    puts "Go to the store and buy some more...99 bottles of beer."
  }
}

Source: 99-bottles-of-beer.net

Argument parsing

See itfParseArgv

Naming schemes

Variables have to fit to the pattern [_a-zA-Z][_a-zA-Z0-9]*.
Functions have to fit to the pattern [^ \t\n\r\f]+.
Variables and function are scrunched together by camelCase or CamelCase.

Community

I think one indicator for the quality of the community is it’s size. And the community will grow, when the language gets more popular.

According to Ohloh, TCL is quite unpopular:

Monthly Commits
Monthly Contributors
Monthly Lines of Code
Monthly Projects

You can also take a look at Google Trends:

Google Trends

The TIOBE Index ranked Tcl on place 43 in March 2014.
lang-index ranked Tcl on place 84 in the general category and on place 34 in the script category.
There are 2,490 Tcl questions on StackOverflow. Compare that to 286,244 questions about Python or 609,648 questions about Java.
Tcl is on rank 18 for GitHub (source).

Execution Speed

Speed comparisons of programming languages are difficult. One nice way to compare the execution speed of programming languages is the benchmarksgame. But sadly, they don’t have TCL.

Additional information

Panasonic Lumix DMC-TZ41

info@martin-thoma.de (Martin Thoma) — Thu, 27 Mar 2014 22:31:00 +0100

Panasonic Lumix TZ41

The Panasonic Lumix TZ41 is currently the best camera in the compact segment.

The TZ41 offers an excellent 20× zoom that is usable due to optical image stabilization. It is compact, leightweight and has a reasonably-sized battery.

Technical specification

Name	Panasonic Lumix TZ41
Model	Panasonic DMC-TZ41EG-K
Price	259.03 Euro
Dimensions	108.3 × 58.9 × 27.7 mm
Weight	198g
Zoom	20×
optical image stabilization	✔
Battery Power	1250mAh
GPS Sensor	✔
Sensor	1/2.3” MOS Sensor with 18.9 Megapixel
Video	1920 x 1080 full HD video
Wi-Fi	✔
NFC	✔
LCD Monitor	✔
Internal Storage	12 MB

This camera is the same as the ‘TZ40’ which has an official data sheet and digitalkamera.de.

Model name

Panasonic	Meaning	Alternatives
Lumix	Compact cameras
DMC	Digital Media Camera
TZ	Traveler Zoom	SZ, TS, GM, LX
41		40 seems to be the same as 41; 60, …
EG	European Union model
K	Black colored	W = White, R = Red, S = Silver

Critic

Charger

Panasonic doesn’t use the standard european charger / cable combination that is used for smartphones.

Loading works via microUSB, but not via standard charger.

Panasonic does not use standard microUSB2USB cables for charging / data exchange

Software

The software is not available for Linux.

Linux

moose@pc08$ cat /etc/issue
Linux Mint 16 Petra
moose@pc08$ uname -a
Linux pc08 3.11.0-12-generic #19-Ubuntu SMP Wed Oct 9 16:20:46 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

and the camera as a USB device:

moose@pc08$ lsusb
Bus 001 Device 006: ID 04da:2372 Panasonic (Matsushita) Lumix Camera (Storage mode)

GPS Assist Tool

The GPS Assist Tool seems to update the camera internal GPS information. It works like this:

Connect camera with SD card in it to the computer.
Start gpsasist.exe with wine.

gpsasist.exe
gps-assist-tool
update completed
Settings

LUMIX Map Tool

The LUMIX Map Tool should allow you to copy maps with information about the environment on your camera. It looks like this:

The drive detection for the SD card doesn’t work on Linux as you can see in the second image. So I’ve have written a Linux version of that program which can be found on GitHub.

Example photographs

To compare the quality of the Panasonic Lumix DMC-TZ41 I have shot some photographs with my old Casio Lumix Exilim EX-Z200.

Macro photographs

Casio
TZ41
TZ41

Normal range photographs

Casio
TZ41
Casio
TZ41
Casio
TZ41
Casio
TZ41

Long-range photographs

Casio
TZ41
Casio
TZ41
Casio
TZ41

Conclusion

Overall, the Panasonic Lumix TZ41 is a great camera. But the lack of a standard charger and a standard cable as well as the missing Linux software is a downer. This is the reason why I

give it 4/5 stars.</span>

More Information

Good German resources are:

Chip.de: A review with some comparisons to other cameras.
Panasonic Lumix DMC-TZ41 - Mein Fazit: A very detailed video review.

Project Glooseberry

info@martin-thoma.de (Martin Thoma) — Tue, 25 Mar 2014 12:32:00 +0100

Blender is an open source software that allows you to create animations. The Blender Foundation is a non-profit organization that helps to develop blender.

To improve Blender, they have created movies. In that process, developers and movie makers sat in one room. So when the movie makers had a problem or discovered a bug, the developers could see and eventually fix that.

By now, they have create four short movies. These movies are DRM-free, free to use and … well, you can basically do what you want with them:

Project Glooseberry is an attempt to create a full length movie. Here is a first trailer:

But creating movies is quite cost intensive. A lot of people are involved and a lot of processing power is needed to create those animations. So the Blender Foundation has started a Crowdfunding Campaign.

You can pledge what you want. I’ve pledged 5 Euros.
When you pay via PayPal and they don’t manage to get the 500 000 Euro they aim for, you get your money back.
The campaign ends on April 19th 2014.

Current campaign status

You can see the current campaign status here.

By now (25th of March, 13:37) there are 809 people that supported this campaign and 65,780 Euro were pledged.

Let me know in comments when you supported the campaing: support now.

How to use Sublime Text via SSH

info@martin-thoma.de (Martin Thoma) — Thu, 20 Mar 2014 21:09:00 +0100

Sublime Text is the best editor I have ever used. One argument for vim and against SSH could be that you can’t simply use Sublime Text when you’re accessing a computer via SSH. But there is a way!

In the following, I will expain the simplest way how to remote edit files with Sublime Text.

Preparation on your computer

Install and start Sublime Text.
Install the rsub package via Package Controll.
Open ~/.ssh/config. Create it if id does not exist yet. Add the code from below.
Start SSH with ssh myname.

This is how the config file should look like:

Host myname
  Hostname pc123.your.network.com
  RemoteForward 52698 127.0.0.1:52698

You can add more information like User yourusername to this file.

Server-Side steps

Download the rmate script:
curl https://raw.github.com/aurora/rmate/master/rmate > rmate
Execute ./rmate yourfile. It will open in your local Sublime Text!

Improvements

These steps have to be done server-side.

With Root access

It’s not so nice to open files with ~/rmate filename all the time. You can use rmate filename after executing this command:

# This creates a symlink
sudo ln -s ~/rmate /usr/local/bin/

You can use other paths than /usr/local/bin/. Look at your PATH for candidates:

echo $PATH

Without Root access

When you don’t have root access, you can’t create a symlink for most (eventually even all) folders in your PATH. But you can expand your PATH:

mkdir -p ~/bin        # create folder if it doesn't exist
ln -s ~/rmate ~/bin/  # create symlink

Now expand your PATH so that it includes ~/bin. There are at least two ways to do so:

You can directly edit your shells .rc file (e. g. .bashrc, .zshrc, .cshrc, …) or
you can edit your .profile

As many shells source .profile I’ll explain this way. First, open ~/.profile. Then add

# First check if that folder exists
if [ -d "$HOME/bin" ] ; then
    # Add your home folder to the end of the current path
    PATH="$PATH:$HOME/bin"
fi

Interpreters and Shells

info@martin-thoma.de (Martin Thoma) — Sat, 08 Mar 2014 11:49:00 +0100

Should you ever be in the position to write a shell or interpreter I hope you will make sure the following things work. Take it as a quality guide. They are ordered by level of importance. The first thing is the most basic one that has to work, the last one is less necessary, but much cooler if you support it.

In the following, I’ll only talk about shells. But most of it will also apply to interpreters.

Level 0: Robustness

A shell has to be robust. Users rely on it when GUI doesn’t work. This means it should definitely not fail. Never.

And just to make sure that you get me correct. Commands get “Entered”. So the user pushes “enter” when he wants something to happen. Everything the shell does meanwhile should not change the system (except for shell-related stuff) and not be able to slow down / crash the system.

Level 1: Speed

I expect a shell to start without recognizable delay. I’m not too sure how fast ‘without recognizable delay’ means. In the most extreme case it would be about 1/100 of a second, because when monitors have 100Hz they are said to be flicker free. You could not even see that. But I think it is not necessary.

Another measure would be reaction time. I’ve just did an online test and saw that my reaction time is about 0.2 seconds. So a shell should be ready for user input after this time.

Level 2: Left / Right arrow keys

When I enter a command, it happens that I forget something. In this case I want to be able to navigate with left / right arrow keys through the console.

Level 3: History with up / down arrow keys

When you use the up-array, you should get the last command you’ve entered. When you press it twice, you get the second last command… So the shell should save your last commands in a so called “history”. This history should be at least

Level 4: Customization

Promt
History length

Level 5: Path autocompletion

The path should autocomplete when you hit Tab. The autocomplete should work as follows:

The autocomplete should never get farer than one folder.
If there are multiple possibilities to autocomplete, then it should only autocomplete what is in common. After a second Tab it should display the possibilities and after a third Tab the shell should go to the first possibility so that you can hit enter to use this possibility.

Level 6: Command autocompletion

The autocomplete function should also complete commands.

Level 7: Fuzzy autocompletion

When you make a typo in a path and hit Tab the shell should correct the typo if possible.

Additional stuff

Some stuff is nice to have, but not really essential:

Some default commands like help, time, pwd, cd and echo.
Ctrl + D as a shortcut for exiting.
Ctrl + C should stop the current command.
The configuration file should be stored in the home folder of the user and it should be called .[name]rc. The dot makes the file invisible by convention and ‘rc’ means ‘resource configuration’.
The prompt configuration should be easy. Some patterns that are used quite often are
- \w for the working directory, where ``$HOME gets abbreviated with ~`
- \u the username
- \h the hostname
- Using colors for different parts
Navigation with Pos 1 and End should work.

Ranking

Here is how some shells rank. Please note, that it’s very difficult to check if a shell is robust. :

Level 0: Windows XP / Windows 7 default shell; Windows Power shell
The shell is too slow. I don’t know if this is still a problem in Windows 8, but I guess so.

The scala interactive interpreter is slow.

Level 1: csh prints “^[[D” when I press the left arrow and “^[[C” when I press the right arrow. When I press Tab it only prints tab. What a crap.

Level 3: python seems to have possibilities to execute arbitrary python code at startup by specifying the environment variable PYTHONSTARTUP, but somehow this does not work on my system.
However, customizing the prompt is fairly easy:

import sys
sys.ps1 = "-->"

Level 4: GHCi, a Haskell compiler, has mastered level 4. The prompt can be configured via ~/.ghci by adding :set prompt "ghci> ".

If you want to show the current path, you can do it like this (source):

let cur fill = do { cwd <- System.Directory.getCurrentDirectory; return (":set prompt \"" ++ cwd ++ fill ++ " \""); }
:def doprompt (\_ -> cur ">")
:def mycd (\dir -> System.Directory.setCurrentDirectory dir >> cur ">")
:doprompt

Level 4.5: bash is robust, takes about 0.11 seconds to start, has a history of 500 lines as you can verify with echo $HISTSIZE, is customizable with .bashrc`.

tcsh takes about 0.02 seconds to start, has a default history size of 100 lines as you can verify with echo $history, is customizable with .tcshrc and .cshrc.

Rubys interactive interpreter irb seems to be fast enough, has a history, the prompt can be configured in ~/.riplrc (source). I don’t know if the history length is limited and can be adjusted.

All shells in this level have path autocompletion as described in 5.1, but do not have autocompletion as described in 5.2.

Level 7: ZSH is the best shell I have ever used. Especially with oh-my-zsh.

Additional resources

How to change your shell prompt: A list of files and commands that might be useful.

Internet at KIT

info@martin-thoma.de (Martin Thoma) — Thu, 06 Mar 2014 11:35:00 +0100

This article is about how to get internet at KIT with Linux. It was tested on Linux Mint 16 MATE which is based on Ubuntu which is based on Debian.

WLAN

At KIT are a lot of WLANs, but only two are important: wkit-802.1x and eduroam.

Mode: Infrastructure
Security: WPA & WPA2 Enterprise
Authentificaiton: Protected EAP (PEAP)
Anonymous identity: anonymous@kit.edu
CA certificate: deutsche-telekom-root-ca-2.crt (source)
PEAP version: Automatic
Inner authentification: MSCHAPv2
Username:
- for wkit-802.1x: uabcd (Your username. It begins with ‘u’ and has 5 letters)
- for eduroam: uabcd@student.kit.edu
Password: (Your password)
IPv4 Settings: Automatic (DHCP)
IPv6 Settings: Ignore

VPN

I use Juniper VPN.

The following lines install some prerequesites, download Juniper from this page, unpack it and execute the shell script.

sudo apt-get install libc6-i386 lib32z1 lib32nss-mdns
wget http://www.scc.kit.edu/scc/sw/juniper/7.4R8/linux_vpn_7.4R8.tar.gz
tar -xzvf linux_vpn_7.4R8.tar.gz
cd juniper_linux
./vpn-install.sh

After you have done this, you can use juniper with

jnc -n kit

The -n flag disables GUI. It will show this:

Server certificate verified and CN is vpn.kit.edu. Saving in /home/moose/.juniper_networks/network_connect/config/vpn.kit.edu.der.
Password: 
Connecting to vpn.kit.edu : 443.
Waiting for ncsvc for 3 seconds... done
ncsvc is running, but tunnel is not established yet. Waiting for 3 seconds... done
ncsvc is running, but tunnel is not established yet. Waiting for 3 seconds... done.
ncsvc is running in background (PID: 11180):
tunnel interface tun0, addr: 141.3.192.60

You can stop it with jnc stop.

Resources

Sublime Text

info@martin-thoma.de (Martin Thoma) — Sat, 01 Mar 2014 17:01:00 +0100

Sublime Text is the coolest editor I have ever used. It has a lot of features, is blazingly fast as I expect it from every editor and has a convenient configuration. It is available for Linux, Windows and Mac. You can use it for free without any restrictions as long as you want. But keep in mind that somebody had to develop this nice software.

Installation

You can get a free version from sublimetext.com or you could install it on Linux Mint via

sudo apt-get install sublime-text

I’ve added a symlink to make it easier to call it from command line:

sudo ln -s /opt/sublime_text/sublime_text /usr/local/bin/sublime

The editor is usable right after the installation, but you might want to make some fine-tuning.

Package Control

The Package Control plugin should definitely be installed. It makes installation of other packages so much easier. After you have installed it, you can press Ctrl+Shift+P to get this dialog:

Sublime Tabs

Configuration

Sublime Text offers plenty of configuration options. You can apply them to projects, users or system wide. Most of the time, I change my preferences for me via Preferences > Settings - User:

Preferences > Settings - User

Here is what I have changed:

{
    "WrapPlus.break_on_hyphens": false,
    "color_scheme": "Packages/Colorsublime - Themes/textmate.tmTheme",
    "detect_indentation": false,
    "draw_white_space": "all",
    "fold_buttons": true,
    "font_face": "Ubuntu Mono",
    "font_size": 15,
    "highlight_line": true,
    "rulers":
    [
        79,
        120
    ],
    "scroll_speed": 0,
    "search_threshold": 1000000,
    "tab_size": 4,
    "translate_tabs_to_spaces": true,
    "use_tab_stops": false
}

Note that you have to install the color scheme and the font (source) to use it.

Command Palette

You can get to the command palette by Ctrl + Shift + P. This opens such a dialog:

automatical alignment

This will do a fuzzy search through all elements in the menu. So you don’t need to use Alt + arrow keys no longer. I love it ☺

Plugins

LaTeXTools

The LaTeXTools package adds support for LaTeX. It adds shortcuts, a pull-down menu when you enter \cref{ and much more.

BracketHighlighter

BracketHighlighter adds brackets on the left side. It looks like this:

Highlight braces

Alignment

Sublime Alignment gives you the possibility to mark text, press Ctrl+Alt+a to align:

automatical alignment

Colorsublime

Colorsublime is a plugin for theming Sublime Text&nbps;3 within seconds. Take a look at colorsublime.com for some examples.

TrailingSpaces

Tools for easy removing trailing spaces with Ctrl + Shift + T. See GitHub repository.

Wrap Plus

Tools for easy wrapping lines with Alt + Q. See GitHub repository.

Python Flake8 Lint

Highlight potential problems with Python code. See GitHub repository.

Themes

First of all, make sure you have installed the Colorsublime package. After you have it, you can press Ctrl + Shift + P and enter “Install Theme”. Then it contacts the server and provides you a list of many good themes. You can go through them with the arrow keys and they will instantly be applied!

I really like the “Textmate” theme, but the “Chrome_DevTools” theme is also good.

Custom Keybindings

You can create custom keybindings via Preferences Key Bindings (User)

I have these:

[
{ "keys": ["ctrl+shift+r"], "command": "reindent", "args": { "single_line": false } },
{ "keys": ["shift+tab"], "command": "unindent", "args": {"single_line":true} },
{ "keys": ["ctrl+7"], "command": "toggle_comment", "args": { "block": false } },
{ "keys": ["ctrl+shift+7"], "command": "toggle_comment", "args": { "block": true } },
{ "keys": ["ctrl+shift+t"], "command": "delete_trailing_spaces" }
]

Custom snippets

Snippets are a very cool feature of ST. They allow you to enter some text, press Tab and get whatever you wanted. So you could create a new .tex document, enter article, press Tab and get a template for a LaTeX document of the article document class.

A tutorial how to create a snippet for the article document class was written Jonathan Page: Creating Snippets in Sublime Text 2 for LaTeX

Buildin Keybindings

Ctrl + p: Goto file

Ctrl + Shift + P: Goto anything

Ctrl + r: Goto section

Ctrl+R in Sublime Text (LaTeX)

Ctrl+R in Sublime Text (Markdown)

Ctrl + Shift + Up / Down: Move the current line one line up / down

Shift + F11: Distraction free mode

Distraction free mode

Ctrl + D: Multi-Select

What could be better

Chrome-like Tabs

Look at this:

Sublime Tabs

Now compare it to this:

Chrome Tabs

Chrome tabs look much cleaner, don’t they? Many others seem to think that, too (source).

Line Wrapping

Sublime Text 3 does wrap points and commas to the next line:

line wrapping

This behaviour is bad and not liked by the community (source).

Support for Tooltips

It would be absolutely great for many plugins like linters if they could make use of tooltips.

Any news on tooltip and sidebar API? Are you planning to implement this (ever? :mrgreen: ) It’s something I very much want to get done, but at this stage I don’t know when that will be.

Source: Sublime Text Forum: Sublime Text 3 Beta

Documentation
colorsublime.com: Lots of themes
tmtheme-editor.herokuapp.com: Adjust your theme online
Sublime Text 2 Video Tutorials: A series of 32 video tutorials for Sublime Text 2. At least the first few are the same in ST 3.

GeoTopo Klausur

info@martin-thoma.de (Martin Thoma) — Tue, 28 Jan 2014 17:39:00 +0100

Dieser Artikel beschäftigt sich mit der Vorlesung „Einführung in die Geometrie und Topologie“ am KIT. Er dient als Prüfungsvorbereitung. Ich habe die Vorlesungen bei Herrn Prof. Dr. Herrlich im Wintersemester 2013/2014 gehört.

Der Artikel wird bis zur Klausur laufend aktualisiert.

Behandelter Stoff

Ist im Skript:

PDF in A4 mit Links zum online anschauen
PDF in A5 zum ausdrucken und binden
Definitionen als PDF in A7 zum Karteikarten-Lernen
LaTeX-Quelldateien

Material

Übungsbetrieb

Wo sind die Übungsblätter: Link</li>
Abgabeform: Über den Kasten im Allianzbau (bei der Mathe-Fachschaft)
Abgabe: Montags
Rücknahme: Im Tutorium
Turnus: wöchentlich, erscheint am Montag
Übungsschein verpflichtend: Es gibt keinen Übungsschein.
Bonus durch Übungsschein: Es gibt keinen Klausurbonus.

Termine und Klausurablauf

Datum: Samstag, den 22. Februar 2014 von 12:00 bis 14:00 Uhr
Ort: Audimax (Quelle)
Punkte: 36
Bestehensgrenze: 14 (?)
Übungsschein: Gibt es nicht.
Bonuspunkte: Gibt es nicht.
Ergebnisse: sind ab heute Nachmittig im Studierendenportal (Stand: 28.02.2014)
Einsicht: Montag, 17. März um 15.00 Uhr im Raum 1C-03 (Stand: 28.02.2014, Quelle)
Erlaubte Hilfsmittel: Keine.
Mitbringen: Studentenausweis und Stift.

Ergebnisse

Im Studierendenportal (Stand: 28.02.2014).

Highend Notebooks

info@martin-thoma.de (Martin Thoma) — Tue, 28 Jan 2014 16:00:00 +0100

My Acer Travelmate 5744Z seems to get broken in soon, so I’m currently looking for a new notebook. As I use my notebook quite often and as this is the third notebook within three years that is not usable any longer (the screen of the first is defect and the graphic card / WLAN of the second does not work propperly with the new Linux kernel), I’m would like to have a high end notebook this time. I hope that I not have to think about notebooks for at least 5 years after that.

Requirements

I work quite a lot with the computer, so the keyboard and the display have to be good. What does good mean? Well, the display has to be at least 15” and as I don’t want to see my pixels any longer it has to have a higher resolution than 1366×768. But it has to be smaller than 30cm × 38cm to fit into my knapsack. A reason why I do not always have my current notebook with me is that it is too heavy with about 2.5kg and does only run for about 3.5h. The new one should be lighter than 2.5kg and run at least 4h.

Ubuntu has to be supported completely, especially WLAN, sound and Bluetooth. Speaking of Bluetooth, I want Bluetooth 4.0, because it introduced a low energy protocol that might be usefull for notebooks. As I live in a city and as I access the internet via my neighbors WLAN (thank you!), I need a good connection: Dual band and 802.11a/b/g/n need to be supported by the notebook. (Dual band is sometimes also called 2x2).

I use it mainly for writing blog articles, LaTeX stuff and some Python programming and watching movies. I download movies via Online TV Recorder which sums up quite soon. Transfering this to my external HDD always takes a lot of time. The new notebook should support USB 3.0 to speed this up. As I don’t have to store anything large, 80 GB SSD is enough. It should be a SSD, because they consume less power, are more durable and are more silent.

I need at least 4GB RAM because … well, did you ever try to use Eclipse for Java+JBoss programming with less than 4GB? I don’t want to have that again.

It would be nice if I could use the notebook outside in the summer which means it has to have a bright matte display.

I want a silent notebook, because I’m very sensitive to noise. My current notebook has less than 30 dB in normal mode (no heavy load). That should set the mark.

Another keyboard-thing: Ctrl has to be at the lower left (→ no thinkpads).

A SD-Card Reader would be nice, but it is not required. Just like a RJ-45 for network cables and a DVD drive you can get it via USB.

I robust case is also important. I had troubles working in the train, because the screen was whipping as hell. But this is not a hard requirement for me as I don’t work in trains that often.

Current Favorites

I’ve looked at a lot of notebooks and I did mainly focus on the technical specification and not at the price (I guess I might work about 10h every day in front of this machine. A good, working notebook is very important to me). However, when I see some notebooks that are ok according to the specs from above, I will take the cheaper one.

By the way, while searching for notebooks I discovered that much of the data on Amazon is wrong. See for example the dimensions of the HP EliteBook 8570p-B6Q03EA-ABD (51.6 x 34.2 x 7.8 cm according to Amazon).

	Asus Zenbook	Samsung Notebook Serie 9
Model	UX51VZ-DB114H	900X4D K01
Price	1380 Euro	995 Euro
Dimensions (B × D × H)	380 × 254 × 20	356.9 × 237 × 14.9 mm
Weight	2.2 kg	1.58 kg
CPU	Intel Core i7 3632QM	Intel Core i5-3337U
Display Size	15.6”	15”
Display Resolution	2880×1620	1600×900
Display more		LED-Display, 400 Nit
Matte display	✘	✔
RAM	8 GB DDR3, 1600 MHz	2× 4 GB DDR3 SDRAM1.600 MHz
Disk	256 SSD	128 GB SSD
Network	?	RJ45 with adapter and 1.000 Mbit/s
Wireless	802.11a/g/n, WiDi	802.11a/b/g/n (2×2), WiDi
WLAN-Chip	?	Intel Wireless-N 7260
Bluetooth	4.0	4.0
Akku	4750 mAh	8200 mAh, up to 10h
USB	3× USB 3.0	2× USB 3.0, 1× USB 2.0
SD Card Reader	✔	✔
Linux-Support	?	partially ([^1],[^2], [^3])
Keyboard	?	Chiclet-keyboard without numblock
Noise	?	29.5 dB in normal mode, 40 dB max

Samsung Series 9
Macbook Pro

The WLAN chipset of the Samsung Serie 9 seems to cause trouble with Linux, but it also seems to be solved by a firmware update.[^5] But one hint seems to be important:

Before you install Linux on a Samsung Serie 9, make sure you update the firmware, because that’s only possible with Windows.

Asus Zenbook seems also to work almost out of the box.[^6] I’ve just learned that you can use

sudo dmidecode -s system-product-name

to determine your exact laptop product name.

In the following, I will give you an overview of the notebooks I took a look at. I think all of them are very good.

In many cases I will need some more equipment:

MicroHDMI 2 VGA adapter: Samsung AA-AH2NMHB/E for 29.90 Euro
USB Ethernet adapter: “Cable Matters - SuperSpeed USB 3.0” for 16 Euro
External DVD burner: Samsung SE-208DB for 30 Euro works with DVD±R Dual layer disks and DVD±RW disks. Is there anything more important to look at?

Acer

	Acer Aspire
Model	V5-573G-54208G50aii
Price	649 Euro
Dimensions (B × D × H)	382 × 256 × 18
Weight	2.04 kg
CPU	Intel Core i5-4200U
Display Size	15.6”
Display Resolution	1920 × 1080
Matte display	✔
RAM	8 GB DDR3, 1600 MHz
Disk	1000 GB HDD
Network	10/100/1000MBit
Wireless
WLAN-Chip	Atheros AR5BWB222
Bluetooth	4.0
Akku	3560 mAh
USB	1× USB 3.0, 2× USB 2.0
SD Card Reader	✔
Linux-Support	✔
Keyboard	Chiclet-Keyboard
Noise	33.3 dB

Apple

	Macbook Pro
Model	Retina 15”
Price	1999 Euro
Dimensions (B × D × H)	358.9 × 247.1 × 18 mm
Weight	2.02 kg
CPU	Intel Core i7 2760QM
Display Size	15.4”
Display Resolution	2880×1800
Display more
Matte display	✔
RAM	8 GB 1600 MHz DDR3L
Disk	256 SSD
Network	no RJ45
Wireless	802.11a/b/g/n
WLAN-Chip	?
Bluetooth	4.0
Akku	up to 7h
USB	2× USB 3.0
SD Card Reader	✔
Linux-Support	?
Keyboard	Chiclet-Keyboard
Noise	29.4 dB in normal mode, 47.4 dB max

Although the hardware looks very nice, it is still comparable with both of my favorites.

Asus

Asus Zenbooks

	Asus Zenbook	Asus Zenbook	Asus Zenbook
Model	UX51VZ-CN035H	UX51VZ-DB114H	U500VZ
Price	1390 Euro	1380 Euro	1350 Euro
Dimensions (B × D × H)	380 × 254 × 20	380 × 254 × 20	380 × 254 × 20 mm
Weight	2.2 kg	2.2 kg	2.2 kg
CPU	Intel Core i7-3612QM	Intel Core i7 3632QM	Intel Core i7-3612QM
Display Size	15.6”	15.6”	15.6”
Display Resolution	1920×1080	2880×1620	1920×1080
Matte display	✔	✘	✔
RAM	8 GB DDR3, 1600 MHz	8 GB DDR3, 1600 MHz	4 GB
Disk	256 SSD	256 SSD	512 GB SSD
Network	10/100/1000, RJ45	?	?
Wireless	802.11a/g/n, 2×2 WiDi	802.11a/g/n, WiDi	802.11a/b/g/n, 2×2
WLAN-Chip	?	?	?
Bluetooth	4.0	4.0	4.0
Akku	?	4750 mAh	4750 mAh
USB	3× USB 3.0	3× USB 3.0	2× USB 3.0
SD Card Reader	✔	✔	✔
Linux-Support	?	?	?
Keyboard	?	?	Chiclet-Keyboard with Numblock
Noise	?	?	34 dB in normal mode, 42 dB max

Asus PU500CA-XO002X and Asus N550JV-CN201H

	Asus	Asus
Model	PU500CA-XO002X	N550JV-CN201H
Price	998 Euro	1099
Dimensions (B × D × H)	383 × 257 × 22.5 mm	383 × 255 × 27
Weight	1.96 kg	2.7 kg
CPU	Intel Core i5-3317U	Intel Core i7-4700HQ
Display Size	15.6”	15.6”
Display Resolution	1366 × 768	1920 × 1080
Matte display	✔	✔
RAM	4 GB	8GB DDR3
Disk	524 GB	1000 GB HDD
Network	?	?
Wireless	802.11 a/b/g/n, WiDi	802.11 b/g/n
WLAN-Chip		Atheros (AR9485)
Bluetooth	4.0	4.0
Akku	4000 mAh, up to 7h	4000 mAh
USB	1× USB 3.0	2× USB 3.0
SD Card Reader	✔	✔
Linux-Support	?	?
Keyboard	Chiclet-Keyboard with Numblock	Chiclet-Keyboard with Numblock
Noise	30.7 dB in normal mode, 38.4 dB max	32.9 dB in normal mode, 38 dB max

Dell

	XPS 15
Model	9530-1906
Price	1711 Euro
Dimensions (B × D × H)	372 × 254 × 18
Weight	2.02 kg
CPU	Intel Core i7-4702HQ
Display Size	15.6”
Display Resolution	3200x1800
Matte display	✘
RAM	8 GB DDR3L, 1600 MHz
Disk	512 GB SSD
Network	(no RJ45)
Wireless	802.11 ac, 2x2
WLAN-Chip	Intel AC 7260
Bluetooth	4.0
Akku	?
USB	3× USB 3.0
SD Card Reader	✔
Linux-Support	?
Keyboard	Chiclet-Keyboard
Noise	29.8 dB, 42.1 dB max

There seem to be other versions of the XPS 15:

XPS 15 (L501X)
XPS 15 (L502X)
XPS 15z

However, I was not able to find any specification of those.

Fujitsu

	Fujitsu Lifebook
Model	E753
Price	1759 Euro
Dimensions (B × D × H)	374 × 374 × 20
Weight	1.99 kg
CPU	Intel Core i7-3632QM
Display Size	15.6”
Display Resolution	1920×1080
Matte display	✔
RAM	8 GB DDR3
Disk	256 GB SSD
Network	10/100/1000MBit
Wireless	802.11 a/b/g/n
WLAN-Chip	Centrino Advanced-N 6235
Bluetooth	4.0
Akku	6700 mAh
USB	3× USB 3.0
SD Card Reader	✔
Linux-Support	?
Keyboard	Chiclet-Keyboard
Noise	33.3 dB

HP

	HP Envy 15
Model	J011SG
Price	811 Euro
Dimensions (B × D × H)	380 × 251 × 28
Weight	2.19 kg
CPU	Intel Core i5-4200M
Display Size	15.6”
Display Resolution	1920×1080
Matte display	✘
RAM	12 GB DDR3L, 1600 MHz
Disk	1000 GB
Network	10/100/1000 RJ-45
Wireless	802.11b/g/n
WLAN-Chip	Intel AC 7260
Bluetooth	4.0
Akku	2200 mAh
USB	4× USB 3.0
SD Card Reader	✔
Linux-Support	?
Keyboard	with numblock
Noise	?

Samsung

Serie 9

The next few lines show the difference of the 900X4C A0A to…

NP900X4C-A01: Intel Core i5-3317U, 128GB SSD, no Dualband
900X3C A03: 3610 mAh, 1399 Euro
900X4B-A01: Intel Core i7 2637M, 1999 Euro
900X4C-A04: Intel Core i5 3317U, 1999 Euro
900X4C-A05: Intel Core i5 3317U, 128GB SSD, no Dualband, ca 1200 Euro
900X4C A06: 256 GB SSD, 15.6” Display, 1629 Euro
900X4C A09: Costs 1999 Euro (any other difference?)
900X4D A03 has also 15” Display, but only 4GB RAM and a Intel Core i5-3317U and a 128 GB SSD. But it costs only 799 Euro.
900X4D K01: Intel Core i5-3337U, 128 GB SSD, 8400 mAh, Dual Band, 997 Euro

Samsung ATIV Book 9 2014 Edition

	Samsung ATIV Book 9 2014 Edition
Model	NP930X5J-K01DE
Price	1599 Euro
Dimensions (B × D × H)	? × ? × 14.9 mm
Weight	1.78 kg
CPU	Intel Core i7-4500U
Display Size	15.6”
Display Resolution	1920×1080
Display more	Touch-Display
Matte display	✘
RAM	8 GB ? MHz
Disk	256GB SSD
Network	RJ45 with adapter and 1.000 Mbit/s
Wireless	802.11 ac (2x2) (source)
WLAN-Chip	Intel Wireless-AC 7260, 802.11 ac
Bluetooth	4.0
Akku	?, 14h
USB	2× USB 3.0, 1 × USB 2.0
SD Card Reader	✔
Linux-Support	?
Keyboard	Chiclet-keyboard
Noise	?

Good audio qualit: 24-bit, 192kHz audio, 2x 2W
HDMI out, mini VGA, an SD card reader
720p webcam

Release date should be 28.03.2014 (source).

Sources

Tuxedo

	Tuxedo Book
Model	BC1503
Price	763 Euro
Dimensions (B × D × H)	374 × 252 × 31 mm
Weight	2.4 kg
CPU	Intel Core i5 4200M
Display Size	15.6”
Display Resolution	1920×1080
Display more	IPS-Display
Matte display	✔
RAM	1×8 GB 1600 MHz
Disk	120 SSD (Samsung EVO / SATA III)
Network	has RJ45 built-in, 10/100/1000
Wireless	802.11 ac/a/b/g/n
WLAN-Chip	Intel Dual AC7260
Bluetooth	4.0
Akku	62,16 Wh, 2.5h[^4]
USB	2× USB 3.0
SD Card Reader	✔
Linux-Support	shipped with Linux Mint ☺
Keyboard	?
Noise	Laut[^4]

TUXEDO Book BU1505 (with Skylake, up to 16GB RAM, ca. 1240 Euro)

Librem 15

	Librem 15
Model
Price	2185 Euro
Dimensions (B × D × H)	375 × 244 × 22 mm
Weight	2.0 kg
CPU	Intel i7-5557U (Broadwell-U architecture)
Display Size	15.6”
Display Resolution	3840×2160
Display more	IPS-Display
Matte display	✔
RAM	1×8 GB 1600 MHz
Disk	250 GB SSD
Network	✘, 10/100/1000
Wireless	802.11 n
WLAN-Chip	?
Bluetooth	✔, ?
Akku	65W, 48 Wh, Up to 6 hours usage
USB	1× USB 3.1, 2× USB 3.0
SD Card Reader	SDXC
Linux-Support	✔
Keyboard	?
Noise	?

Other

On a first glance, the Samsung ATIV Book 8 NP880Z5E-X01 looked quite promising. But it doesn’t have an SSD, it weights 2.54 kg, but has no optical drive.

Samsung ATIV 870Z5E-X03DE
ATIV Book 8 870Z5E X04
ATIV Book 8 880Z5E X01
ATIV Book 9 900X4D K01 (notebookinfo.de)

Notebooks that did not meet the criteria

Display is too small:

Chromebook Pixel has only 12.85”
XPS 13 has only 13.3”
Asus Zenbook has only 13.3”
Lenovo IdeaPad U300s have only 13.3”
All Samsung Series 9 X3A seem to have 13.3” displays
Samsung Series 9 900X3D-A02: 13.3”
Samsung Series 9 900X3C-A01: 13.3”
Samsung Series 9 900X3C A07: 13.3”

Too low resolution:

Acer TravelMate 6594eG-464G50Mikk
Asuspro PU500
All Acer Aspire TimelineU M5

To heavy:

Lenovo IdeaPad Y510p: 2.89 kg
HP EliteBook 8570p-B6Q03EA-ABD: 2.91 kg
Sony Vaio VPC-F21Z1E/BI: 3.17 kg

Availability: seems not to be available on Amazon

Samsung Serie 9 NP900X4C-A02
Sony Vaio SV-E1511V1EW
Sony Vaio VGN-TX2
HP Envy 6-1000sg

Other:

Acer Aspire M3-581TG: Too loud, only 667MHz RAM

Dear Notebook-Producers

After searching so much for notebooks, I have some hints for you what you could do better:

Add a single specification page for each notebook. This page should include at least:
- Weight in kg and dimensions in mm
- Battery life in mAh
- Display (size, glare or matte display, supported resolutions)
- Exact CPU name (not only Intel i5 - if it varies, list all possible CPUs)
- Disk (size, SSD or not)
- Wireless support (IEEE 802.11 supported standards? Dual band? Bluetooth? Bluetooth version?)
- Keyboard: Does it have a numblock? Backlit?
- Does it have a DVD-Player / Burner? Blue-Ray?
- Webcam (resolution)
- Microphone
- Sensors (GPS)
- Image of the notebook
A good specification page would include:
- Noise in dB
- Information about Linux support (especially Debian)
Explain your version names!
Provide a possibility to compare your products like Intel does with ark.intel.com for its processors
Provide a possibility to filter your products by technical specification.
Add an image of your product to Wikipedia Commons

References

[^1] Linux and the Samsung Series 9 NP900X3C: A review for the NP900X3C and openSUSE on 24th or September, 2012. [^2] Samsung Series 9 - Ubuntu Community Page [^3] Linux auf Samsung Series 9 2012 [^4] Tuxedo Book DC1502 im Test [^5] No wireless with Intel Centrino Advanced-N 7260 [^6] AsusZenbook - Ubuntu Community Page

Install and configure computer

info@martin-thoma.de (Martin Thoma) — Sat, 25 Jan 2014 10:14:00 +0100

When I reinstall my computer, I usually do these following steps:

Copy all data to an external HDD
Write down all WLAN configurations (eventually with screenshots; NOT ONLY PASSWORDS!)
Write down all programs that I use.
Export configuration of those programs.
Wait a week or a month and see if someting is missing in the lists from above.
Drop the old system and install a new one

Software I usually install

If possible, I will give the debian package names in the following list:

LaTeX and scientific writing
- jabref: A reference manager
- gnuplot
- pdf2svg
- aspell and aspell-de
Google Chrome
Multimedia
- vlc: A very good DVD player
- OnlineTvRecorder and especially OTR-Verwaltung
  - avidemux wine mplayer
- sudo add-apt-repository ppa:clipgrab-team && sudo apt-get update && sudo apt-get install clipgrab
Graphics
- gimp
- inkscape
- dia
- imagemagick
- pdf2svg librsvg2-bin
Programming
- vim
- python python3 python-numpy python-setuptools python-mysqldb
- ruby ruby-sqlite3 ruby-mysql
- gcc g++ cmake build-essential gdb
- OpenGL: xorg-dev libglu1-mesa-dev freeglut3 freeglut3-dev libglew1.5 libglew1.5-dev libglu1-mesa libglu1-mesa-dev libgl1-mesa-glx libgl1-mesa-dev
- apache2 php5 php5-mysql
- zsh and Oh-my-zsh
- eclipse
- sqlitebrowser
- tcl
- phpmyadmin selfhtml
- meld diffpdf
- virtualbox
Themes
- Balazan-Theme from bisigi-project (simply download it.)
Other
- sublime_text
- libreoffice
- curl
DRM-caused (I want to watch DVDs!)
- ubuntu-restricted-extras libdvd-pkg libdvdread4 libdvdnav4, then run sudo dpkg-reconfigure libdvd-pkg
- totem banshee mplayer rythmbox

Configure

Set standards

update-alternatives --config editor
update-alternatives --config x-www-browser

MATE

See issues/262:

mv ~/.config/gtk-3.0/bookmarks ~/.config/gtk-3.0/bookmarks-backup
ln -s ~/.gtk-bookmarks ~/.config/gtk-3.0/bookmarks

GUI

I like the old menu bar quite a lot. It opens instantly and is customizable:

Old menu bar

You can get it back in MATE by doing a right-click on the menu. Then click on “add to panel”:

Add to panel

After that, the following dialog will pop up. Choose “Menu Bar”

Add menu bar

DRM-Stuff

sudo /usr/share/doc/libdvdread4/install-css.sh
sudo regionset #use that with caution

dotfiles

See github.com/MartinThoma/dotfiles.

Data

Download / copy all data back from GitHub / external HDDs to my internal HDD.

SLURM

info@martin-thoma.de (Martin Thoma) — Fri, 24 Jan 2014 14:58:00 +0100

SLURM is short for ‘Simple Linux Utility for Resource Management’. It helps you to make use of a cluster by giving you a command line interface to add jobs to a queue. That means, you and other users can specify program calls that get executed as soon als all conditions are met.

I currently need to work with this, because I have some pretty heavy tasks that occur periodically, but don’t need to be executed immediately. More concrete, I am working in the field of ASR - Automatic Speech Recognition. In ASR you have models that include probabilities for … well, let’s don’t get into detail for that. The important part is that those models need to be trained. And they can get adapted to speakers.

As soon as you have more recordings, you can try to improve the model. And this training involves a lot of computation and data processing. As the training job is executed periodically, but not always when a new recording is present, you might also have recordings for many different speakers. So you can improve many models when this script is started. So this can be done on different computers in parallel.

SLURM basic usage

A simple call could look like this:

srun python test.py -aflag -param value

Other notable commands are:

squeue: Gives a list of all running / pending jobs
sbatch: Run a batch script in background
scancel: Stop a job

Important parameters for srun are:

--mem=[X in MB]
--job-name=[Name of your SLURM job]
--dependency: Start this job when all dependencies are met. This could be time or other jobs

Another important command is squeue. It allows you to list all jobs in command line (if you have a GUI: sview).

You can get your jobs only with squeue -u mthoma, where mthoma should be replaced by your user name.

Help texts

sbatch

$ sbatch --help
Usage: sbatch [OPTIONS...] executable [args...]

Parallel run options:
  -A, --account=name          charge job to specified account
      --begin=time            defer job until HH:MM MM/DD/YY
  -c, --cpus-per-task=ncpus   number of cpus required per task
      --comment=name          arbitrary comment
  -d, --dependency=type:jobid defer job until condition on jobid is satisfied
  -D, --workdir=directory     set working directory for batch script
  -e, --error=err             file for batch script's standard error
      --export[=names]        specify environment variables to export
      --get-user-env          load environment from local cluster
      --gid=group_id          group ID to run job as (user root only)
      --gres=list             required generic resources
  -H, --hold                  submit job in held state
  -i, --input=in              file for batch script's standard input
  -I, --immediate             exit if resources are not immediately available
      --jobid=id              run under already allocated job
  -J, --job-name=jobname      name of job
  -k, --no-kill               do not kill job on node failure
  -L, --licenses=names        required license, comma separated
  -m, --distribution=type     distribution method for processes to nodes
                              (type = block|cyclic|arbitrary)
  -M, --clusters=names        Comma separated list of clusters to issue
                              commands to.  Default is current cluster.
                              Name of 'all' will submit to run on all clusters.
      --mail-type=type        notify on state change: BEGIN, END, FAIL or ALL
      --mail-user=user        who to send email notification for job state
                              changes
  -n, --ntasks=ntasks         number of tasks to run
      --nice[=value]          decrease secheduling priority by value
      --no-requeue            if set, do not permit the job to be requeued
      --ntasks-per-node=n     number of tasks to invoke on each node
  -N, --nodes=N               number of nodes on which to run (N = min[-max])
  -o, --output=out            file for batch script's standard output
  -O, --overcommit            overcommit resources
  -p, --partition=partition   partition requested
      --propagate[=rlimits]   propagate all [or specific list of] rlimits
      --qos=qos               quality of service
  -Q, --quiet                 quiet mode (suppress informational messages)
      --requeue               if set, permit the job to be requeued
  -t, --time=minutes          time limit
      --time-min=minutes      minimum time limit (if distinct)
  -s, --share                 share nodes with other jobs
      --uid=user_id           user ID to run job as (user root only)
  -v, --verbose               verbose mode (multiple -v's increase verbosity)

Constraint options:
      --contiguous            demand a contiguous range of nodes
  -C, --constraint=list       specify a list of constraints
  -F, --nodefile=filename     request a specific list of hosts
      --mem=MB                minimum amount of real memory
      --mincpus=n             minimum number of logical processors (threads) per node
      --reservation=name      allocate resources from named reservation
      --tmp=MB                minimum amount of temporary disk
  -w, --nodelist=hosts...     request a specific list of hosts
  -x, --exclude=hosts...      exclude a specific list of hosts

Consumable resources related options:
      --exclusive             allocate nodes in exclusive mode when
                              cpu consumable resource is enabled
      --mem-per-cpu=MB        maximum amount of real memory per allocated
                              cpu required by the job.
                              --mem >= --mem-per-cpu if --mem is specified.

Affinity/Multi-core options: (when the task/affinity plugin is enabled)
  -B  --extra-node-info=S[:C[:T]]            Expands to:
       --sockets-per-node=S   number of sockets per node to allocate
       --cores-per-socket=C   number of cores per socket to allocate
       --threads-per-core=T   number of threads per core to allocate
                              each field can be 'min' or wildcard '*'
                              total cpus requested = (N x S x C x T)

      --ntasks-per-core=n     number of tasks to invoke on each core
      --ntasks-per-socket=n   number of tasks to invoke on each socket
      --cpu_bind=             Bind tasks to CPUs
                              (see "--cpu_bind=help" for options)
      --hint=                 Bind tasks according to application hints
                              (see "--hint=help" for options)
      --mem_bind=             Bind memory to locality domains (ldom)
                              (see "--mem_bind=help" for options)


Help options:
  -h, --help                  show this help message
  -u, --usage                 display brief usage message

Other options:
  -V, --version               output version information and exit

squeue

squeue --help
Usage: squeue [OPTIONS]
  -A, --account=account(s)        comma separated list of accounts
                  to view, default is all accounts
  -a, --all                       display jobs in hidden partitions
  -h, --noheader                  no headers on output
      --hide                      do not display jobs in hidden partitions
  -i, --iterate=seconds           specify an interation period
  -j, --job=job(s)                comma separated list of jobs IDs
                  to view, default is all
  -l, --long                      long report
  -M, --clusters=cluster_name     cluster to issue commands to.  Default is
                                  current cluster.  cluster with no name will
                                  reset to default.
  -n, --nodes=hostlist            list of nodes to view, default is 
                  all nodes
  -o, --format=format             format specification
  -p, --partition=partition(s)    comma separated list of partitions
                  to view, default is all partitions
  -q, --qos=qos(s)                comma separated list of qos's
                  to view, default is all qos's
  -s, --step=step(s)              comma separated list of job steps
                  to view, default is all
  -S, --sort=fields               comma separated list of fields to sort on
      --start                     print expected start times of pending jobs
  -t, --states=states             comma separated list of states to view,
                  default is pending and running,
                  '--states=all' reports all states
  -u, --user=user_name(s)         comma separated list of users to view
  -v, --verbose                   verbosity level
  -V, --version                   output version information and exit

Help options:
  --help                          show this help message
  --usage                         display a brief summary of squeue options

More information

Ruby

info@martin-thoma.de (Martin Thoma) — Wed, 15 Jan 2014 23:11:00 +0100

I’m currently learning Ruby to improve my Jekyll blog. At the moment, I can’t really say if I like Ruby.

Just like PHP and Python, Ruby is implicitly typed. This means you can do the following:

# A list
a = [1,2,3,4,5]
# An integer
b = 42
# A floating point number
c = 3.141

Ruby is also dynamically typed:

a = 42
a = "now a is a string!"

One thing that is pretty annoying is the usage of begin ... end. I thought LaTeX was the only “language” in use that had this syntax.

Syntax examples

If, else if, else

a = 1

if a == 1 || a % 2 == 0
    puts "a equals 1 or is even"
elsif a == 3
    puts "a equals 3"
else
    puts "a does not equal 1 or 3 and is not even"
end

String manipulation

String manipulation is weird in Ruby. Take a look at the last example. What would you expect?

s = "0123456789"
puts s[2..4]
puts s[2..s.length()]
puts s[2..-2]

Output is:

234
23456789
2345678

Now what would you expect to happen with "0123456789"[2..4]? I would expect Ruby to give "234". But this one actually fails.

Dictionaries

Dictionaries, which are sometimes also called ‘maps’ or ‘associative arrays’ are a neat data structure. Ruby uses this syntax:

dictionary = {"a"=> 7, "b"=> "x"}

if dictionary.has_key?("a")
    puts "key 'a' is in dictionary! It is " + dictionary["a"].to_s
    dictionary["a"] = 42
    puts "Now it is " + dictionary["a"].to_s
    puts dictionary.size()
    puts dictionary.length()
    puts dictionary.count()
end

map

myList = [1,2,3,4,5]
x = myList.map{|element| [element,element+1]}
x.each do |a, b|
    puts "a: " + a.to_s + ", b: " + b.to_s
end

will print

a: 1, b: 2
a: 2, b: 3
a: 3, b: 4
a: 4, b: 5
a: 5, b: 6

.count(), .length(), .size()

At a first glance, those three seem to be aliases. But .count() is not quite the same as .length() and .size() (source).

Blocks

A very interesting idea in Ruby is that of blocks. It seems to be similar to Pythons decorators:

def time_method(method=nil, *args)
  beginning_time = Time.now
  if block_given?
    yield
  else
    self.send(method, args)
  end
  end_time = Time.now
  puts "Time elapsed #{(end_time - beginning_time)*1000} milliseconds"
end
 
time_method do
  (1..10000).each { |i| i }
end

Logging

Logging in Ruby is very convenient. See Logger Documentation

Notable libraries and projects

Nokogiri: Parsing HTML

The following code will parse yourHTMLcode and select all a-tags (also known as links):

require 'nokogiri'

doc = Nokogiri::HTML.parse(yourHTMLcode)
links = doc.css('a').map { |link| [link['href'],link.text] }
links.each do |link, linktext|
    puts "Link '"+link+"' was used with '"+linktext+"'"
 end

Sequel

This way you can select elements from a SQLite-Database:

require 'sequel'

db = Sequel.sqlite("search.db")
rows = db.fetch( "SELECT rowid FROM pages WHERE url='/imprint';" ).all
#puts rows
#puts rows.count()
puts rows[0][:rowid]

See also:

Sum of cubed digits riddle

info@martin-thoma.de (Martin Thoma) — Sat, 04 Jan 2014 14:53:00 +0100

Let $N \in \mathbb{N}$ be a number with digits $a_k$ , where $a_0$ is the least significant digit and $n$ is the most significant digit.

Find all numbers with the following property:

$$N = \sum_{k=0}^n a_k \cdot 10^k = \sum a_k^3$$

Lower and upper bounds

The lower bound is $0$
The upper bound is $2916$ since $4\cdot 9^3 = 2916$

You can find better upper bounds, but as we’ve just reduced the possible number space to only 2917 numbers, it doesn’t really matter.

Find all solutions

#!/usr/bin/env python3

def find_sum_of_cubes():
    """Returns all numbers N with the following property
        N = \sum_{k=0}^n a_k \cdot 10^k = \sum a_k^3
    """
    def has_sum_of_cubes_property(n):
        digits = map(int, list(str(n)))
        return sum(map(lambda n: n**3, digits)) == n
    return list(filter(has_sum_of_cubes_property, range(2917)))

if __name__ == "__main__":
    print(find_sum_of_cubes())

which gives

[0, 1, 153, 370, 371, 407]

How to use Jekyll with GitHub

info@martin-thoma.de (Martin Thoma) — Mon, 09 Dec 2013 00:00:00 +0100

You’ve probably noticed that I didn’t write any posts the last few weeks. The reason is that I’ve migrated my WordPress Blog to Jekyll. This means it takes some source files and generates purely static pages from that. The generation process is independant of user requests.

Jekyll is a static blog generator, just like Pelican, Hyde, nanoc and Octopress.

I’ve spend about 40-80 hours to migrate from WordPress to Jekyll. And I’m not done jet.

Jekyll compared with WordPress

Jekyll is a static site generator. This means you have the source files on your computer. Then you generate the website with Jekyll and upload only the generated files. So you only push content to the server, but you don’t have to download anything from the server.

Reasons for Jekyll:

Security: With Jekyll, you only upload static files (HTML, CSS, JavaScript, Images, …). There is nothing where the user can pass some data. This also means there is one thing less to update.
Assuming your provider updates your server software (e.g. Apache) you don’t have to update anything. In contrast, when you don’t regularly install updates on your WordPress blog (and hope that those updates don’t break anything), your blog is quite likely to be hacked.
Speed: The webserver needs only to serve the sites. Nothing else. Of course, when you use caching with PHP you might get into a similar situation with WordPress. But it will never be faster.
Hosting: You only need webspace. This reduces hosting cost significantly. Additionally, you can use Amazon S3 for hosting!
Backups: Creating security backups is VERY easy with Jekyll. Every tool that can make bakups of files can backup your Jekyll blog. No need to worry about databases. If a file is corrupt, only that file is affected. No worries about maximum execution time for importing / exporting backups. No need to get SSH access. Simple FTP access does the job.

Reasons for WordPress:

Comments: There is no way to get comments only with static pages. So you need something else, e.g. disqus.
Search: Could probably be done with JavaScript, but I guess it is difficult.
Editor: WordPress gives you a WYSIWYG editor. I don’t know if there is something similar for Jekyll.
Compile time: Compilation time is very long for Jekyll. For my blog, it needs over 6 minutes. As I test the result quite often before I publish posts, this is very annoying.
Tagging, Category pages, Author pages: Currently, Jekyll lacks basic support for blogging. You don’t have tag pages per default, the plugins that provide tags don’t have paginated tag index pages. The same problem occurs when it comes to categories or authors.
Timed posts: I did not use timed posts very often, but it is very easy to create them with WordPress. With Jekyll, on the other hand, you have to know how to create cronjobs. And your computer has to be running.

Install Jekyll

On an Ubunty system I need for this blog:

$ sudo apt-get install ruby1.9.1-dev imagemagick ruby-rmagick libmagickwand-dev ruby-execjs ruby-nokogiri
$ sudo gem install jekyll
$ sudo gem install dimensions
$ sudo gem install fileutils
$ sudo mkdir -p /var/www/blog

GitHub

Create your repository

Go to github.com, sign in and create a new repo:
Call it [Username].github.io.

Branches

The way to use Jekyll with GitHub is by using branches. Go to your Git repository that cointains your blog:

moose@pc08 ~/Downloads/MartinThoma.github.io $ ls
_config.yml  favicon.ico  index.html  Makefile  Readme.md
css          images       js          _plugins  _site
_drafts      _includes    _layouts    _posts    thumbs

Now you should create a new branch that will contain your source files. The following command creates a branch sources that starts where the branch master currently is:

git checkout -b source master

Now update this branch to the server:

git push -u origin source

When you enter

git checkout . master

you will switch to the master brach. The same way you can switch to the source branch. After you’ve entered the command, you can look at the folder in your file system. There will only be the data of the current branch.

Custom Domain

If you want to host your content at GitHub, but have a custom Domain like martin-thoma.com instead of martinthoma.github.io, you have to:

Ask your provider (in my case “Knallhart”) to add an A-record to Github.
Add a file called CNAME with content martin-thoma.com (yes, without http://) to the root of your directory

GitHub also offers some help on setting up a custom domain with Pages.

FTP Server

If you have your own FTP server, you probably want to use it.

One tool that might now come to your mind is rsync. But rsync needs SSH (source). If you have SSH, then you can do something like this:

rsync -avz --delete _site/ user@host:/path/to/web/root

The --delete options “delete[s] extraneous files from dest dirs”, -a means archive which preserves the owner, group, change date and some more of files, -v is verbose as always and -z is for compression while the file transfer happens. --progress might also be interesting, especially for the first upload.

Otherwise, you might want to try curlftpfs. This program lets you mount a FTP folder:

curlftpfs ftp.example.com/backups /mnt/ftpserver

and you can also umount it:

rsync --delete /var/backups /mnt/ftpserver

Markdown

I’ve switched between rdiscount and redcarpet. The former is faster, the latter supports fenced code blocks. I finally stuck with redcarped, because Liquid has problems when it comes to C++ for loops after curly braces.

Other Markdown parsers are maruku (which has a Multiple lines for HTML <li>-tag issue) and kramdown (which is slow and does not handle fenced code blocks and LaTeX correctly).

Similar to Matthias I would like to share a feature matrix of Markdown parsers included into Jekyll. I have used jekyll 1.4.3 to create the following table. I’ve used this page to see if fenced code blocks and pygments is working and that page for LaTeX. Everything was tested with this site.

	`redcarpet`	`rdiscount`	`kramdown`	`maruku`
LaTeX	✓	✓	✓	?
Fenced code blocks	✓	✓	~	?
Pygments	✓	✓	✓	?
Site generation	176.90s	165.41s	293.26s	?

Kramdown destroyed some fenced code blocks (but not all) and maruku did not even compile my site at all.

Linebreaks and newline

Linebreaks are an issue. Sometimes I want to get a <br/>, sometimes I make linebreaks to make reading of the text files easier. It’s basically this discussion.

Currently, I’m not satisfied with the situation. I never had that problem with WordPress. WordPress simply created paragraphs just at the right location.

Customization

You can create custom Liquid filters, plugins and templates. Everything is quite easy. The Liquid templating language seems to be very similar to Django Templates (Python).

Make the Website super-fast

CSS Minification

sudo gem install juicer
juicer install jslint
juicer install yui_compressor

Images

I’ve included small images as base64 (used this online tool). According to caniuse it’s quite save to use.

Site Search

Site search is a real problem. I’ve seen three solutions so far:

Dynamic Search: You can add a dynamic part to your statically generated website. For example, I create a search/index.php and a SQLite database like it was described here.
Static JavaScript: Create a JSON file or something similar and search dynamically with JavaScript in it.
External Search Engines: You could use a search engine for searching your site, of course.
- Hosted by you:
- Commercial
  - Google Custom Search
  - Index Den: Has no direct support to parse your website

PHP+SQLite

JavaScript solutions

One JavaScript solution I’ve found is lunr. This one is really bad as it copies the whole body to a json file. This json file has to be loaded before it works. But my posts total at the moment to 2MB. I’m pretty sure my readers don’t want to wait until 2MB are downloaded. So this one does only work for smaller websites.

Christan Fei’s solution does only search in the title and category.

Templates

Jekyll uses Liquid as a templating language. It is similar to Django templates. Here is a short introduction to Liquid.

A Sublime Text template is the following:

<snippet>
    <content><![CDATA[
---
layout: post
title: ${1:}
author: Martin Thoma
date: 2014-11-22 17:19
categories: 
- ${2:Cyberculture}
tags: 
- ${3:Rating}
featured_image: logos/${4:star.png}
---
${5:}
]]></content>
    <!-- Optional: Set a tabTrigger to define how to trigger the snippet -->
    <tabTrigger>---</tabTrigger>
    <!-- Optional: Set a scope to limit where the snippet will trigger -->
    <!-- <scope>source.python</scope> -->
</snippet>

Some tests

Validation
- validator.w3.org: My site is HTML-valid. The error that this validator shows is caused by an bug inside of the validator itself.
- jigsaw.w3.org: My site is CSS-valid.
Speed: Could be better…
- tools.pingdom.com
- PageSpeed 72 on mobile and 85 on desktop
Accessiblity:
- wave.webaim.org
- Functional Accessibility Evaluator
More:
- Mobile readyness
- Load test
- Alexa
- Google Structured Data Testing Tool: Test if google can extract the author from your blog posts
- Twitter card validator (more about Twitter cards)
- Nibbler: This one tests quite a lot.

I’ve also used LinkChecker to check if all new links are valid. I’ve found quite a lot of old links and replaced them with new links.

Resources

How to check if a point is inside of a polygon?

info@martin-thoma.de (Martin Thoma) — Mon, 18 Nov 2013 21:36:09 +0100

Suppose you have a with $n$ sides. This is called a $n$ -glon.

Basics about polygones

A $n$ -glon can be defined by a list of $n$ points.

Note that the order is important:

[A, B, C, D, E, F, G] != [A, B, C, D, F, E, G]

I will not consider self-intersecting polygones for the following statements. I’m aware of them, but whenever you have a self-intersecting polygon you can create multiple polygones that cover the same area and don’t intersect each other (some pairs might have a finite number of points in common, but not an infinite number).

Is a point in a triangle / a rectangle

It is quite easy to check weather a point is inside of a triangle or inside of a rectangle. I have already written an article about how to check if a point is inside of a rectangle.

Is a point inside of a n-glon?

Let $P$ be a point and $N = [P_1, P_2, \dots, P_n]$ be a $n$ -glon. It is now much more difficult to check if $P$ is inside of $N$ . The area-approach works for convex $n$ -glons, but that’s it.

Count Crossing Line Segments

However, you can try another approach which I have visualized in the following image:

Check if P is inside of N

When $P$ is inside of $N$ , every line $P_{1}P, P_{2}P, \dots, P_{n}P$ will cross the polygon lines $P_{1}P_2,P_{2}P_3, \dots, P_{n}P_1$ an even number of times. If P is outside, at least one of the lines $P_{i}P$ will cross a polygon line $P_{j}P_{j+1}$ once.

This means, for every check you have to check $n^2$ pairs of line segments for crossings. How you can do that is explained in my article How to check if two line segments intersect.

This algorithm is in $\mathcal{O}(n^2)$ time complexity (it does need a constant amount of additional space).

Triangularization

When you have a lot of querys, you might want to divide your polygon into convex polygones. The easiest way to do this might be dividing $N$ into triangles.

That way, you can check for every triangle if $P$ is inside of it. I assume that the number of triangles is not bigger than $n$ . As the check is in constant time for one triangle, you would have an algorithm that needs $\mathcal{O}(n)$ time and space for its checks (+ some preprocessing which is done only once).

Word Error Rate Calculation

info@martin-thoma.de (Martin Thoma) — Fri, 15 Nov 2013 08:59:15 +0100

The Word Error Rate (short: WER) is a way to measure performance of an ASR. It compares a reference to an hypothesis and is defined like this:

[\mathit{WER} = \frac{S+D+I}{N}]

where

S is the number of substitutions,
D is the number of deletions,
I is the number of insertions and
N is the number of words in the reference

Examples

REF: What a bright day
HYP: What a day

In this case, a deletion happened. “Bright” was deleted by the ASR.

REF: What a day
HYP: What a bright day

In this case, an insertion happened. “Bright” was inserted by the ASR.

REF: What a bright day
HYP: What a light day

In this case, an substitution happened. “Bright” was substituted by “light” by the ASR.

Range of values

As only addition and division with non-negative numbers happen, WER cannot get negativ. It is 0 exactly when the hypothesis is the same as the reference.

WER can get arbitrary large, because the ASR can insert an arbitrary amount of words.

Calculation

Interestingly, the WER is just the Levenshtein distance for words.

I’ve understood it after I saw this on the German Wikipedia:

[ \begin{align} m &= |r|
n &= |h|
\end{align} ]

[ \begin{align} D_{0, 0} &= 0
D_{i, 0} &= i, 1 \leq i \leq m
D_{0, j} &= j, 1 \leq j \leq n \end{align}]

[ \text{For } 1 \leq i\leq m, 1\leq j \leq n
D_{i, j} = \min \begin{cases} D_{i - 1, j - 1}&+ 0 \ {\rm if}\ u_i = v_j
D_{i - 1, j - 1}&+ 1 \ {\rm(Replacement)}
D_{i, j - 1}&+ 1 \ {\rm(Insertion)}
D_{i - 1, j}&+ 1 \ {\rm(Deletion)} \end{cases} ]

But I have written a piece of pseudocode to make it even easier to code this algorithm:

WER calculation

Python

#!/usr/bin/env python

def wer(r, h):
    """
        Calculation of WER with Levenshtein distance.
        Works only for iterables up to 254 elements (uint8).
        O(nm) time ans space complexity.

        >>> wer("who is there".split(), "is there".split())
        1
        >>> wer("who is there".split(), "".split())
        3
        >>> wer("".split(), "who is there".split())
        3
    """
    # initialisation
    import numpy
    d = numpy.zeros((len(r)+1)*(len(h)+1), dtype=numpy.uint8)
    d = d.reshape((len(r)+1, len(h)+1))
    for i in range(len(r)+1):
        for j in range(len(h)+1):
            if i == 0:
                d[0][j] = j
            elif j == 0:
                d[i][0] = i

    # computation
    for i in range(1, len(r)+1):
        for j in range(1, len(h)+1):
            if r[i-1] == h[j-1]:
                d[i][j] = d[i-1][j-1]
            else:
                substitution = d[i-1][j-1] + 1
                insertion    = d[i][j-1] + 1
                deletion     = d[i-1][j] + 1
                d[i][j] = min(substitution, insertion, deletion)

    return d[len(r)][len(h)]

if __name__ == "__main__":
    import doctest
    doctest.testmod()

Explanation

No matter at what stage of the code you are, the following is always true:

If r[i] equals h[j] you don't have to change anything. The error will be the same as it was for r[:i-1] and h[:j-1]
If its a substitution, you have the same number of errors as you had before when comparing the r[:i-1] and h[:j-1]
If it was an insertion, then the hypothesis will be longer than the reference. So you can delete one from the hypothesis and compare the rest. As this is the other way around for deletion, you don't have to worry when you have to delete something.

Formatting Strings in Python

info@martin-thoma.de (Martin Thoma) — Thu, 07 Nov 2013 20:05:33 +0100

In Python, you can use the following ways to format Strings:

Print directly

Printing them directly (just like printf in C):

birthday = 28
month = "April"
year = 1990
print("My birthday is the %i-th %s %i." % (birthday, month, year))

The first string contains the rules how to format. %i means that the first argument in the following tuple should be interpreted as a integer. The second one %s should be interpreted as a string and the third one again as a integer.

Save as string

>>> a = "Why is %i the answer?" % 42
>>> a
'Why is 42 the answer?'

Named formatting

You might prefer named formatting:

>>> "{guy} loves {girl}.".format(girl="Marie", guy="Martin")
'Martin loves Marie.'

You can also store this first in a dictionary an unpack it:

>>> myDictionary = {"girl":"Marie","guy": "Martin","other":"Internet"}
>>> "{guy} loves {girl}.".format(girl="Marie", guy="Martin")
'Martin loves Marie.'

Date and Time

You can format time any way you like, just look at this reference.

Lists

Question: I would like to print a list! How do I do that? Answer: Convert your list to a string

>>> myList = [1,2,3]
>>> print("Your list: %s" % (str(myList)))
Your list: [1, 2, 3]

str and repr

When you build your own objects, you should add an implementation for the method __str__ and __repr__. The first one should return a string representation that is human readable of the object, the second one should return a string that identifies the object.

Formatters

`%i`	Integer	`>>> print("%i" % (123)) 123`
`%s`	String	`>>> print("%s" % ("Martin")) Martin`
`%o`	Int as octal	`>>> print("%o" % (123.123)) 173`
`%x`	Int as hexadecimal (lower case)	`>>> print("%x" % (123.123)) 7b`
`%X`	Int as hexadecimal (upper case)	`>>> print("%X" % (123.123)) 7B`
`%f`	Floating point	`>>> print("%f" % (123.123)) 123.123000`
`%.2f`	Floating point with two decimal places	`>>> print("%.2f" % (123.123)) 123.12`
`%e`	Floating point in scientific notation	`>>> print("%e" % (123.123)) 1.231230e+02`
`%%`	Percent sign	`>>> print("%i%%" % (65)) 65%`
`%6.2f`	Print a float with 2 decimal places. Add spaces if this has less than 6 characters.	`>>> print("%6.2f" % (65.123)) 65.12`

Columns

my_list = [('Easybox 1234', 54, 'DC:9F:DB:B2:B1:1C'),
           ('FRITZ!Box 6360 Cable', 12, '24:65:11:06:71:54'),
           ('wkit-802.1x', 15, 'A0:D3:C1:9F:FF:11')]
header = u"{0:<20}{1:>6}{2:>20}".format('SSID',
                                        'Signal',
                                        'HwAddress')
print(header)
print("-"*len(header))
for ssid, signal, hwaddress in my_list:
    print(u"{0:<20}{1:>6}{2:>20}".format(ssid,
                                         str(signal)+'%',
                                         hwaddress))

results in

SSID                Signal           HwAddress
----------------------------------------------
Easybox 1234           54%   DC:9F:DB:B2:B1:1C
FRITZ!Box 6360 Cable   12%   24:65:11:06:71:54
wkit-802.1x            15%   A0:D3:C1:9F:FF:11

Resources

Fibonacci, recursion and decorators

info@martin-thoma.de (Martin Thoma) — Thu, 31 Oct 2013 11:08:01 +0100

I think everybody who learned something about recursion has seen the Fibonacci sequence:

$ f(n) := \begin{cases} n &\text{if } n \leq 1\\ f(n-1) + f(n-2) &\text{otherwise} \end{cases} $

The simplest solution to get this number is:

def fib(n):
    if n < 2:
        return n
    else:
        return fib(n-1) + fib(n-2)

The problem is, of course, that the number of evaluations goes wild. Here is a table of the number of function calls

n	0	1	2	3	4	5	6	7	8	9	10	20
calls	1	1	3	5	9	15	25	41	67	109	177	21891

To be exact, the number of calls of the fib-function is:

$ f(n) := \begin{cases} 1 &\text{if } n \leq 1\\ f(n-1) + f(n-2) + 1 &\text{otherwise} \end{cases} $

This means the dumb function is in $\mathcal{O}(2^n)$ ! (I’m not quite sure, but this I think this is not only time complexity, but also space complexity. I think it is not tail recursive, so the complete stackframe has to be saved.)

Memorization with decorators

One way to solve the problem much faster (in fact in $\mathcal{O}(n)$ time and space complexity) by storing values we already calculated.

A very neat way to achieve this are decorators. It might be a common problem that you have a recursive, mathematical function with no side effects. So you can write a wrapper that checks if the value has already been calculated. If not, the function proceeds as usual. It it has already been calculated, you can simply look it up:

def memoize(obj):
    cache = {}

    def memoizer(*args, **kwargs):
        if args not in cache:
            cache[args] = obj(*args, **kwargs)
        return cache[args]
    return memoizer

@memoize
def fib(n):
    if n < 2:
        return n
    else:
        return fib(n-1) + fib(n-2)

Notice that I’ve only added @memoize over the function definiton of fib! I love Python ☺

By the way, this formula has also some limitations. Python has a fixed maximum recursion depth. So fib(332) worked fine, but fib(333) gave:

RuntimeError: maximum recursion depth exceeded in comparison

You can get around this limitation by successive calls of fib:

# Call to fill array
fib(332)

# The number of recursive steps is now much smaller:
print(fib(500))

That gave 139423224561697880139724382870407283950070256587697307264108962948325571622863290691557658876222521294125. A pretty big number.

Formula of Moivre-Binet

The formula of Moivre-Binet gives a closed form for calculating fibonacci numbers:

$\varphi = \frac{\sqrt{5}+1}{2}$ $\psi = 1 - \varphi$ $f(n) = \frac{\varphi^n - \psi^n}{\phi - \psi}$

Although this is mathematically exact, it will not work on computers due to a fixed floating point precision. Lets check how long it works:

#!/usr/bin/env python

import functools
def memoize(obj):
    cache = {}

    @functools.wraps(obj)
    def memoizer(*args, **kwargs):
        if args not in cache:
            cache[args] = obj(*args, **kwargs)
        return cache[args]
    return memoizer

@memoize
def fib(n):
    if n < 2:
        return n
    else:
        return fib(n-1) + fib(n-2)

def moivreBinet(n):
    phi = (5**0.5+1)/2
    psi = 1 - phi
    return int((phi**n - psi**n)/(phi - psi))

from itertools import count
for i in count(0):
    exact = fib(i)
    constTime = moivreBinet(i)
    if exact != constTime:
        print(("The %i-th fibonacci number is %i. Moivre-Binet "
             + "gives due to precicion error %i (delta=%i).") 
                 % (i, exact, constTime, abs(exact-constTime)))
        break

So the answer is:

The 72-th fibonacci number is 498454011879264. Moivre-Binet gives due to precicion error 498454011879265 (delta=1).

This is a reason to prefer the $\mathcal{O}(n)$ solution over the $\mathcal{O}(1)$ solution. If you’re only exact for 72 numbers, you could also simply store them. Looking number up form an array is always faster than any calculation.

Very high numbers

The following solution is fast and works 0.075 seconds for the 20000 Fibonacci number (which has 4180 digits).

def fib(n):
    def accFib(n, Nm2=0, Nm1=1):
        for i in range(n):
            Nm2, Nm1 = Nm1, Nm1+Nm2
        return Nm2   
    return accFib(n)

Additional ressources

The article on literate programs is worth reading. They show some very different programs that calculate Fibonacci numbers.

Programmierparadigmen Klausur

info@martin-thoma.de (Martin Thoma) — Thu, 24 Oct 2013 12:46:16 +0200

Dieser Artikel beschäftigt sich mit der Vorlesung „Programmierparadigmen“ am KIT. Er dient als Prüfungsvorbereitung. Ich habe die Vorlesungen bei Herrn Prof. Dr. Snelting im Wintersemester 2013/2014 gehört.

Behandelter Stoff

Vorlesung

Datum	Kapitel	Inhalt
23.10.2013	Funktionale Programmierung 10	Haskell: Quicksort, Listen, Filter, Cons-Operator
25.10.2013	Funktionale Programmierung 11	Haskell: filter, map, iter, foldr, foldl, Currying, Extensionalitätsprinzip, Kombinatoren (Summe, Produkt), flatten, cons, zip
30.10.2013	Funktionale Programmierung 12	Haskell: zipWith, short circuit evaluation, foldl, foldr, Unendliche Listen, Typen, Polymorphie
06.11.2013		Backtracking, Algebraische und rekursive Datentypen, map for trees, Typklassen
08.11.2013		Typklassen, Monaden
13.11.2013		Sichtbarkeitsbereich `$\subseteq$` Gültigkeitsbereich; `$\alpha$` / `$\eta$`-Äqivalenz, Redex; Funktion, die sich als eigenes Argument nimmt; `$\lambda$`-Klakül ist Turing-Mächtig
29.11.2013	Logische Programmierung	Prolog
10.01.2014	Scala	Kein `;`, weniger verbose als Java, ...
15.01.2014	Scala	Concurrency in Scala: Actors, react; MPI, OpenMP
22.01.2014	-	X10: async, val, var
24.01.2014	-	C (volatile)

Übungsblätter

Übungsblatt	Inhalt
ÜB 0: Haskell	Haskell installieren (siehe UbuntuUsers), Maximum dreier Zahlen berechnen
ÜB 1: Rekursive Funktionen in Haskell	Potenzen, Primzahlen, Sortieren

Tutorium

16.12.2013

let f = \ x.plus x x in f (f c_2)    ^= (\ f. f (f c_2)) (\x. plus x x)

let wird wegen dem Typsystem benötigt (let ist polymorph, $\lambda$ -Term nicht).

let f = \x.1 in (f 7) + (f["a"])    ^= (\ f.   ) (\ x. 1)
    f: \alpha_5 -> int
    f: \forall \alpha_5. \alpha_5 \rightarrow int

Material

Inoffizielles Skript in A5 (LaTeX-Quellen): Wer das gerne für ca. 10 Euro in SW gedruckt mit Ringbindung hätte, soll mir eine E-Mail schreiben
Vorlesungswebsite und Übungsblätter
Ein Anki-Deck (NICHT meines!)
Stackexchange:

Klausurvorbereitung

H-99: Ninety-Nine Haskell Problems
P-99: Ninety-Nine Prolog Problems
Scala
- S-99: Ninety-Nine Scala Problems
- Learning Scala by Joel Abrahamsson

Übungsbetrieb

Wo sind die Übungsblätter: Link
Abgabeform: auf Papier und via E-Mail
Abgabe: Datum steht auf den Übungsblättern; Ort: Kasten im Keller des Infobaus
Rücknahme: im Tutorium
Turnus: wöchentlich, erscheint am Donnerstag.
Übungsschein verpflichtend: Es gibt keinen Übungsschein.
Bonus durch Übungsschein: Es gibt keinen Klausurbonus.

Termine und Klausurablauf

Siehe Klausurtermine-Seite für zukünftige Termine (z.B. Programmierparadigmen am 23.09.2014)

Datum: Donnerstag, den 10. April 2014 von 14:00 bis 16:00 Uhr (Quelle)
Ort: Audimax (Quelle)
Punkte: 120
Punkteverteilung: Vermutlich etwas in dieser Richtung:

25 Punkte: Haskell / Scala
20 Punkte: Logische Programmierung
25 Punkte: Typinferenz / Lambda-Kalkül
10 Punkte: C
10 Punkte: MPI
10 Punkte: X10
20 Punkte: Compilerbau

Bestehensgrenze: ?
Übungsschein: Gibt es nicht.
Bonuspunkte: Gibt es nicht.
Ergebnisse: stehen seit dem 17.04.2014 fest
Einsicht: am Mittwoch den 30.04.2014, 14:00 Uhr - 16:00 in Raum 010, Informatik Gebäude (Geb. 50.34) (bekanntgegeben über Vorlesungswebsite am 17.04.2014)
Erlaubte Hilfsmittel: (siehe Website)

Erlaubte Hilfsmittel für die Klausur sind alle Quellen in Papierform, insbesondere

Vorlesungsfolien der Veranstaltung Programmierparadigmen

Übungszettel und Beispiellösungen der Veranstaltung Programmierparadigmen

Bücher, Ausdrucke und beliebige eigenen Aufzeichnungen

Ergebnisse

Stehen seit dem 17.04.2014 fest.

Kritik

In der Vorlesung werden zu viele Inhalte behandelt. Eine Folge ist, dass nichts richtig behandelt werden. Außerdem erscheinen mir einige der Inhalte weder in der Wissenschaft noch in der Wirtschaft relevant zu sein (Prolog, X10). Ich würde vorschlagen diese Inhalte zu entfernen. Wenn man der Meinung ist, dass man Parallelität mehr behandeln sollte, dann wäre vermutlich CUDA deutlich wichtiger als X10.

Working with Terminal

info@martin-thoma.de (Martin Thoma) — Tue, 22 Oct 2013 10:40:46 +0200

I’ve just switched from Bash to ZSH because of oh-my-ZSH. I think this is just the right time to explain the words Shell, command line, Terminal, Bash and ZSH.

Terminal is an terminal emulator, sometimes also called a “terminal window”. I work in a window environment (MATE) and I want to use command line tools within that environment. So I need a “terminal window”:

Terminal Window with ZSH and Bash

ZSH and Bash are both Unix shells. A shell is a command line interpreter that provides a text-based user interface.

Command line describes the textual way you interact with the computer. When you’re in a graphical user interface situation you interact by manipulating windows with your keyboard/mouse. When you’re in a text-based user interface situation, you interact by entering commands in a line (hence command line).

Solarized Dark Theme

The Solarized Dark Theme is very good for command line. It can be installed like this as explained here:

git clone https://github.com/oz123/solarized-mate-terminal.git
cd solarized-mate-terminal
./solarized-mate.sh dark

restart the terminal.

Oh-my-ZSH Installation

Oh-my-ZSH is a plugin for ZSH. I think this plugin is very good and makes a big difference to Bash. So when you look at the screenshots below, keep in mind that this is not a “plain vanilla” zsh.

Install “Oh-my-ZSH”
Install “powerline fonts” and change your Terminal font to one of them
Change your Terminal theme to “agnoster” by setting ZSH_THEME="agnoster" in ~/.zshrc
Set your terminal theme to “Solarized Dark” (description)
Make ZSH your default Shell in MATE Terminal (description) and eventually sudo chsh -s /usr/bin/zsh username

ZSH and Bash

Here are some differences. On the left side is zsh, on the right is bash:

Bash vs zsh: cd command completion
Bash vs zsh: Git prompt indicator
Bash vs zsh: Spelling correction
Bash vs zsh: time command

I like the time command of bash more, but that’s it. All other interactions are either almost the same or better in zsh. I especially like that zsh doesn’t print everything again when you autocomplete with tab. And it also autocompletes when you make an capitalization error.

I also begin to like the Git-specific prompt indicators:

ZSH 'git add' indicator

Some usefull tools

ack

You might already know grep. And if you’ve worked with it, you might already have typed something like the following:

grep --exclude-dir=".svn" "searchterm" *
grep -rI "onlytextSearchterm" .

An alternative to grep is ack (for Ubuntu users: ack-grep). See beyondgrep.com.

Windows

It seems to be possible to get something similar (the same?) for Windows. See OH MY CYGWIN.

Part III: Matrix multiplication on multiple cores in Python, Java and C++

info@martin-thoma.de (Martin Thoma) — Mon, 21 Oct 2013 16:39:35 +0200

This is Part III of my matrix multiplication series. Part I was about simple matrix multiplication algorithms and Part II was about the Strassen algorithm. Part III is about parallel matrix multiplication.

We got some pretty interesting results for matrix multiplication so far. Now, I would like to get to know in how far performance increases (or decreases) if I make use of multiple cores. I only have two cores, so I hope somebody with a better computer will also run the ikj single core algorithm form part I and the parallel version from this article and post the results as a comment.

The implementations

As last time, I’ve added the scripts to a GIT repository. So you can test it on your machine.

Before we start implementing code for multiple processors, we have to get an algorithm that is actually parallelisable. You could use Cannon’s algorithm, a algorithm that makes use of systolic arrays or try to find a solution by your own. The Scalable Universal Matrix Multiplication Algorithm (short: SUMMA) could also work. The paper that I’ve linked is well-written and easy to understand. You should definitively read it, if you’re interested in matrix multiplication.

I will not use any advanced algorithm in this article. I will make the outer most for loop of the ikj-algorithm (see part I) execute in parallel.

More about parallel matrix multiplication:

Python

Because of global interpreter lock (GIL), you need to start new processes in Python (source).

The ikj single core algorithm implemented in Python needs:

time python ikjMultiplication.py -i 2000.in > 2000-nonparallel.out

real	36m0.699s
user	35m53.463s
sys	0m2.356s

The most simple way to parallelize the ikj algorith is to use the multiprocessing module and compute every line of the result matrix C with a new process. But for the 2000x2000-example, this would mean we started 2000 processes. The overhead is much worse than the benefit:

time python ikjMultiplication.py -i 2000.in > 2000-parallel.out

real	20m47.693s
user	40m34.460s
sys	0m2.092s

When we share memory, the code looks like this:

#!/usr/bin/python
# -*- coding: utf-8 -*-

import multiprocessing, numpy, ctypes

def read(filename):
    lines = open(filename, 'r').read().splitlines()
    A = []
    B = []
    matrix = A
    for line in lines:
        if line != "":
            matrix.append(map(int, line.split("\t")))
        else:
            matrix = B
    return A, B

def printMatrix(matrix, f):
    for line in matrix:
        f.write("\t".join(map(str,line)) + "\n")

def lineMult(start):
    global A, B, mp_arr, part
    n = len(A)
    # create a new numpy array using the same memory as mp_arr
    arr = numpy.frombuffer(mp_arr.get_obj(), dtype=ctypes.c_int)
    C = arr.reshape((n,n))
    for i in xrange(start, start+part):
        for k in xrange(n):
            for j in xrange(n):
                C[i][j] += A[i][k] * B[k][j]

def ikjMatrixProduct(A, B, threadNumber):
    n = len(A)
    pool = multiprocessing.Pool(threadNumber)

    pool.map(lineMult, range(0,n, part))
    # mp_arr and arr share the same memory
    arr = numpy.frombuffer(mp_arr.get_obj(), dtype=ctypes.c_int) 
    C = arr.reshape((n,n))
    return C

def extant_file(x):
    """
    'Type' for argparse - checks that file exists but does not open.
    """
    if not isfile(x):
        raise argparse.ArgumentError("{0} does not exist".format(x))
    return x

if __name__ == "__main__":
    import argparse, sys
    from os.path import isfile
    from argparse import ArgumentParser

    parser = ArgumentParser(description="ikjMatrix multiplication")
    parser.add_argument("-i", "--input",
        dest="filename", required=True, type=extant_file,
        help="input file with two matrices", metavar="FILE")
    parser.add_argument("-o", "--output",
        type=argparse.FileType(mode='w'),
        default=sys.stdout, dest="output",
        help="file to write output to (default=stdout)")
    args = parser.parse_args()

    A, B = read(args.filename)
    n, m, p = len(A), len(A[0]), len(B[0])

    threadNumber = 2
    part = len(A) / threadNumber
    if part < 1:
	    part = 1

    # shared, can be used from multiple processes
    mp_arr = multiprocessing.Array(ctypes.c_int, n*p)
    C = ikjMatrixProduct(A, B, threadNumber)
    printMatrix(C, args.output)

and it needs MUCH more time:

time python ikjMultiplication-shared.py -i 2000.in > 2000-parallel-2threads.out

real	131m35.433s
user	250m36.820s
sys	0m9.533s

When we don’t use shared memory, things run faster:

#!/usr/bin/python
# -*- coding: utf-8 -*-

import multiprocessing, numpy, ctypes

def read(filename):
    lines = open(filename, 'r').read().splitlines()
    A = []
    B = []
    matrix = A
    for line in lines:
        if line != "":
            matrix.append(map(int, line.split("\t")))
        else:
            matrix = B
    return A, B

def printMatrix(matrix, f):
    for line in matrix:
        f.write("\t".join(map(str,line)) + "\n")

def lineMult(start):
    global A, B, C, part
    n = len(A)
    for i in xrange(start, start+part):
        for k in xrange(n):
            for j in xrange(n):
                C[i][j] += A[i][k] * B[k][j]

def ikjMatrixProduct(A, B, threadNumber):
    n = len(A)
    pool = multiprocessing.Pool(threadNumber)

    pool.map(lineMult, range(0,n, part))
    return C

def extant_file(x):
    """
    'Type' for argparse - checks that file exists but does not open.
    """
    if not isfile(x):
        raise argparse.ArgumentError("{0} does not exist".format(x))
    return x

if __name__ == "__main__":
    import argparse, sys
    from os.path import isfile
    from argparse import ArgumentParser

    parser = ArgumentParser(description="ikjMatrix multiplication")
    parser.add_argument("-i", "--input",
        dest="filename", required=True, type=extant_file,
        help="input file with two matrices", metavar="FILE")
    parser.add_argument("-o", "--output",
        type=argparse.FileType(mode='w'),
        default=sys.stdout, dest="output",
        help="file to write output to (default=stdout)")
    args = parser.parse_args()

    A, B = read(args.filename)
    n, m, p = len(A), len(A[0]), len(B[0])

    threadNumber = 2
    part = len(A) / threadNumber
    if part < 1:
	    part = 1

    C = [[0 for i in xrange(n)] for j in xrange(n)]
    C = ikjMatrixProduct(A, B, threadNumber)
    printMatrix(C, args.output)

time python ikjMultiplication.py -i 2000.in > 2000-parallel-4threads.out

real	22m46.066s
user	41m42.396s
sys	0m2.324s

Java

Shell.java:

import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;
import java.util.LinkedList;
import java.util.List;
import java.util.ArrayList;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;

public class Shell {
	static List<ArrayList<ArrayList<Integer>>> read(String filename) {
		ArrayList<ArrayList<Integer>> A = new ArrayList<ArrayList<Integer>>();
		ArrayList<ArrayList<Integer>> B = new ArrayList<ArrayList<Integer>>();

		String thisLine;

		try {
			BufferedReader br = new BufferedReader(new FileReader(filename));
			// Begin reading A
			while ((thisLine = br.readLine()) != null) {
				if (thisLine.trim().equals("")) {
					break;
				} else {
					ArrayList<Integer> line = new ArrayList<Integer>();
					String[] lineArray = thisLine.split("\t");
					for (String number : lineArray) {
						line.add(Integer.parseInt(number));
					}
					A.add(line);
				}
			}

			// Begin reading B
			while ((thisLine = br.readLine()) != null) {
				ArrayList<Integer> line = new ArrayList<Integer>();
				String[] lineArray = thisLine.split("\t");
				for (String number : lineArray) {
					line.add(Integer.parseInt(number));
				}
				B.add(line);
			}
			br.close();
		} catch (IOException e) {
			System.err.println("Error: " + e);
		}

		List<ArrayList<ArrayList<Integer>>> res = new LinkedList<ArrayList<ArrayList<Integer>>>();
		res.add(A);
		res.add(B);
		return res;
	}

	static void printMatrix(int[][] matrix) {
		for (int[] line : matrix) {
			int i = 0;
			StringBuilder sb = new StringBuilder(matrix.length);
			for (int number : line) {
				if (i != 0) {
					sb.append("\t");
				} else {
					i++;
				}
				sb.append(number);
			}
			System.out.println(sb.toString());
		}
	}

	public static int[][] parallelMult(ArrayList<ArrayList<Integer>> A,
			ArrayList<ArrayList<Integer>> B, int threadNumber) {
		int[][] C = new int[A.size()][B.get(0).size()];
		ExecutorService executor = Executors.newFixedThreadPool(threadNumber);
		List<Future<int[][]>> list = new ArrayList<Future<int[][]>>();

		int part = A.size() / threadNumber;
		if (part < 1) {
			part = 1;
		}
		for (int i = 0; i < A.size(); i += part) {
			System.err.println(i);
			Callable<int[][]> worker = new LineMultiplier(A, B, i, i+part);
			Future<int[][]> submit = executor.submit(worker);
			list.add(submit);
		}

		// now retrieve the result
		int start = 0;
		int CF[][];
		for (Future<int[][]> future : list) {
			try {
				CF = future.get();
				for (int i=start; i < start+part; i += 1) {
					C[i] = CF[i];
				}
			} catch (InterruptedException e) {
				e.printStackTrace();
			} catch (ExecutionException e) {
				e.printStackTrace();
			}
			start+=part;
		}
		executor.shutdown();

		return C;
	}

	public static void main(String[] args) {
		String filename;
		int cores = Runtime.getRuntime().availableProcessors();
		System.err.println("Number of cores:\t" + cores);
		
		int threads;
		if (args.length < 3) {
			filename = "3.in";
			threads = cores;
		} else {
			filename = args[1];
			threads = Integer.parseInt(args[2]);
		}

		List<ArrayList<ArrayList<Integer>>> matrices = read(filename);
		ArrayList<ArrayList<Integer>> A = matrices.get(0);
		ArrayList<ArrayList<Integer>> B = matrices.get(1);
		int[][] C = parallelMult(A, B, threads);
		printMatrix(C);
	}
}

LineMultiplier.java:

import java.util.ArrayList;
import java.util.concurrent.Callable;

public class LineMultiplier implements Callable<int[][]> {
	ArrayList<ArrayList<Integer>> A;
	ArrayList<ArrayList<Integer>> B;
	int start;
	int end;
	public int[][] C;

	public LineMultiplier(ArrayList<ArrayList<Integer>> a,
			ArrayList<ArrayList<Integer>> b, int s, int e) {
		A = a;
		B = b;
		C = new int[a.size()][b.get(0).size()];
		start = s;
		end = e;
	}

	@Override
	public int[][] call() {
		for (int i = start; i < end; i++) {
			for (int k = 0; k < B.size(); k++) {
				for (int j = 0; j < B.get(0).size(); j++) {
					C[i][j] += A.get(i).get(k) * B.get(k).get(j);
				}
			}
		}
		return C;
	}
}

Execute it with only one thread:

time java Shell -i 2000.in 1 > 2000-paralllel.out
Number of cores:	2
0

real	0m40.571s
user	0m42.259s
sys	0m0.388s

Execute it with two threads:

time java Shell -i 2000.in 2 > 2000-paralllel.out
Number of cores:	2
0
1000

real	0m30.188s
user	0m54.999s
sys	0m0.512s

Note that real time is lower than user time. The reason is simply that the execution time on each processor is added. So user time might be double as high as real time!

We got from 40.571s down to 0m30.188s!

Information for parallelism

C++

Making the ikj-algorithm parallel is trivial with C++. You only need to add #pragma omp parallel for before the outer most for loop and add -fopenmp as a compile flag! (If you really want to see the code, go to my Git repository.)

$ time ./ikj-algorithm.out 2 2000.in > 2000-parallel.out

real	0m12.563s
user	0m20.569s
sys	0m0.156s

So we got from 20.407 seconds down to 12.563 seconds by adding only one line!

More Information

Guide into OpenMP: Easy multithreading programming for C++

Part IV: How to multiply matrix with its transpose in Python and C++

info@martin-thoma.de (Martin Thoma) — Fri, 18 Oct 2013 17:41:13 +0200

This is Part IV of my matrix multiplication series. Part I was about simple implementations and libraries: Performance of Matrix multiplication in Python, Java and C++, Part II was about multiplication with the Strassen algorithm and Part III will be about parallel matrix multiplication (I didn't write it yet).

You can always multiply a matrix $J \in \mathbb{R}^{n \times m}$ with its transpose $J^T$ , because $J^T \in \mathbb{R}^{m \times n}$ . You will get a matrix $C \in \mathbb{R}^{n \times n}$ .

Standard matrix multiplication of square matrices $\in \mathbb{R}^{n \times n}$ is in $\mathcal{O}(n^3)$ . With the Strassen algorithm you can multiply in $\approx \cal O(n^{2.807})$ . But this is for general matrix multiplication. When we do $J \cdot J^T$ we have more structure, so it might be possible to do this multiplication faster.

One important property of the result matrix $R = J \cdot J^T$ is symmetry. So $R_{i,j} = R_{j,i}$ . If we used the ikj-algorithm for this multiplication, we needed $n^2 \cdot m$ operations. This way, we only need $\frac{n^2 +n}{2} \cdot m$ operations. Yes, I know, asymptotically it is irrelevant. But skipping almost half of the operations is still quite good.

Python

NumPy

I guess doing this with NumPy is the best option:

#!/usr/bin/python
# -*- coding: utf-8 -*-
 
import numpy

from optparse import OptionParser
parser = OptionParser()
parser.add_option("-i", dest="filename", default="2000.in",
     help="input file with two matrices", metavar="FILE")
(options, args) = parser.parse_args()
 
def read(filename):
    lines = open(filename, 'r').read().splitlines()
    J = []
    for line in lines:
        J.append(map(int, line.split("\t")))
    return numpy.matrix(J)
 
def printMatrix(matrix):
    matrix = numpy.array(matrix)
    for line in matrix:
        print "\t".join(map(str,line))

J = read(options.filename)
R = J * J.T
printMatrix(R)

Time:

real	7m19.223s
user	7m12.147s
sys	0m2.388s

When you want to do this in an application, you might want to use numpy.load.

C++

First try

#include <sstream>
#include <string>
#include <fstream>
#include <iostream>
#include <vector>
#include <algorithm>

using namespace std;

int getMatrixN(string filename) {
    std::ifstream inFile(filename.c_str()); 
    return std::count(std::istreambuf_iterator<char>(inFile), 
             std::istreambuf_iterator<char>(), '\n');
}

int getMatrixM(string filename) {
	string line;
	ifstream infile;
	infile.open (filename.c_str());
	getline(infile, line);
	return count(line.begin(), line.end(), '\t') + 1;
}

void read(string filename, vector< vector<double> > &A) {
	string line;
	FILE* matrixfile = freopen(filename.c_str(), "r", stdin);

	int i = 0, j;
    double a;
	while (getline(cin, line) && !line.empty()) {
		istringstream iss(line);
		j = 0;
		while (iss >> a) {
			A[i][j] = a;
			j++;
		}
		i++;
	}

	fclose (matrixfile);
}

vector< vector<double> > ikjalgorithmTranspose(
                                   vector< vector<double> > &J, 
								   vector< vector<double> > &T,
								   vector< vector<double> > &R, 
                                   int n, int m) {
	for (register int i = 0; i < n; i++) {
		for (register int k = 0; k < m; k++) {
			for (register int j = i; j < n; j++) {
				R[i][j] += J[i][k] * T[k][j];
			}
		}
	}

	for (register int i = 0; i < n; i++) {
		for (register int j = 0; j < i; j++) {
			R[i][j] += R[j][i];
		}
	}

	return R;
}

void transpose(vector< vector<double> > &A, 
               vector< vector<double> > &B, int n, int m) {
    for (int i=0; i < n; i++) {
        for (int j=0; j < m; j++) {
            B[j][i] = A[i][j];
        }
    }
}

void printMatrix(vector< vector<double> > &matrix, int n) {
	for (int i=0; i < n; i++) {
		for (int j=0; j < n; j++) {
			if (j != 0) {
				cout << "\t";
			}
			cout << matrix[i][j];
		}
		cout << endl;
	}
}

int main (int argc, char* argv[]) {
	string filename;
	if (argc < 3) {
		filename = "../Testing/5161x7058.in";
	} else {
		filename = argv[2];
	}

	int n = getMatrixN(filename);
    int m = getMatrixM(filename);
	vector<double> inner (m);
	vector<double> inner2 (n);
	vector< vector<double> > J(n, inner), T(m, inner2), R(n, inner);
	read (filename, J);
    transpose(J, T, n, m);
	ikjalgorithmTranspose(J, T, R, n, m);
	printMatrix(R, n);
	return 0;
}

Time:

real	5m31.488s
user	5m27.560s
sys	0m1.812s

Direct multiplication

One might think that transposing first is a bad idea, because you can do this:

vector< vector<double> > ikjDirect(vector< vector<double> > &J, 
								   vector< vector<double> > &R, 
                                   int n, int m) {
	for (register int i = 0; i < n; i++) {
		for (register int k = 0; k < m; k++) {
			for (register int j = 0; j < n; j++) {
				R[i][j] += J[i][k] * J[j][k];
			}
		}
	}
	return R;
}

I stopped execution after 15 minutes.

How to install Arch Linux 2013.10.01

info@martin-thoma.de (Martin Thoma) — Mon, 07 Oct 2013 03:19:05 +0200

I’ve watched the following video which explains pretty much everything there is to know:

You might also want to read the Arch Linux Installation Guide. The German guide is also very good.

I just like to add some notes. They are primarily for myself, so you really should read the installation guide or watch the video.

The Arch Linux installation setup is command line based.

Keyboard layout

You can switch to a German keyboard layout with this command:

loadkeys de

Partitioning and mounting

You need a bootable partition and a swap partition (that is at least as big as much RAM you have). Partitioning can be done with cfdisk (configure disk).

Check the result with

parted /dev/sda print
fdisk -l

Afterwards:

mkfs.ext4 /dev/sda1
mkfs.ext4 /dev/sda3
mkswap /dev/sda2
swapon /dev/sda2
mount /dev/sda1 /mnt
mkdir /mnt/home
mount /dev/sda3 /mnt/home/

Getting internet

I only have WLAN at the moment, which makes things more difficult:

wpa_passphrase "SSID" PASSWORD > /etc/wpa_supplicant/wpa_supplicant.conf
wpa_supplicant -i wlp4so -D wext -c /etc/wpa_supplicant/wpa_supplicant.conf -B
dhcpcd wlp4s0

wlp4s0 is the wlan interface (which is wlan0 most of the time). You get it with ip link.

I had a lot of fun at this point as you can see here.

Install base system

packstrap -i /mnt base base base-devel

After accepting all defaults (by pressing enter) and letting it download:

genfstab -U -p /mnt  :  sed 's/rw,realtime,data=ordered/defaults,realtime/' >> /mnt/etc/fstab
arch-chroot /mnt
nano /etc/locale.gen

Uncomment your language (from now on, your promt should be “sh-4.2#”).

locale=gen
echo LANG=en_US.UTF-8 > /etc/locale.conf
export LANG=en_US.UTF-8
ln -s /usr/share/zoneinfo/Europe/Berlin /etc/localtime
hwclock --systohc --utc
systemctl enable dhcpcd.service

Now edit and uncomment multilib. Then:

mkinitcpio -p linux

Now set the password by entering passwd. Create your user:

useradd -m -g users -G wheel -s /bin/bash moose
passwd moose

Synchronise caches:

pacman -Syy

Install grub:

pacman -S grub-bios
pacman -S os-prober
grub-install /dev/sda
grub-mkconfig -o /boot/grub/grub.cfg
exit
restart

Give yourself sudo:

pacman -S sudo
nano /etc/sudoers

Add your user to the bottom section “User privilege specification”

Install GNOME

pacman -S gnome xorg
systemctl enable gdm.service

MATE

Add the following to your /etc/pacman.conf:

[mate]
SigLevel = Optional TrustAll
Server = http://repo.mate-desktop.org/archlinux/$arch

Run

# pacman -Syy
# pacman -S mate mate-extras

Lets see if I get an answer to some essential basic questions.

Projects I never realized

info@martin-thoma.de (Martin Thoma) — Tue, 01 Oct 2013 23:41:37 +0200

The following is a collection of ideas for projects I had, but never realized. I would really love to do them, but they seem to be a little bit too time consuming to do them in my free time. Please send me an E-Mail if you would like to realize them!

Book portal

I really miss a book recommendation portal. It should allow you to mark books you’ve read or started to read, let you rate and tag books. The tags should be created by users, similar to the system StackExchange uses. Tags might be funny”, “zombie”, “magic”, “romance”, “love”, … and users should rate for tags for books. In some general settings you define language(s) you know. Every book, which should be administrated by ISBN number if possible, should have information about the language. The portal should also see when books are only released with a new cover / collectors edition and notice that its the same story.

With this information, it should recommend books and allow you to search books.

Eventually you could connect to friends and let their ratings influence what you get.

And, very important, it should let you follow series and/or authors. So you should be able to say “When there is a new book of the ‘Harry Potter’ series, send me an e-mail!” or “When there is a new book of the ‘Harry Potter’ series translated to ‘German’, send me an e-mail!”.

Science and Education Platform

Sometimes, scientists get new insights that are able to influence millions. But until such a great invention or discovery is made, hundreds or thousands of people might have thought about the same problem. Today, with MOOCs education in some fields is quite open to a lot of people. Khan academy offers many very basic courses, Coursea and Udacity a few advanced ones. But the process of creating new content seems to be quite closed. Wikiversity is more open, but very limited. For example, I think it is not possible to include my graphic filter examples. And it is not possible to track progress of students.

I think it is necessary to gamify this. Both, students and educators, should get rewards. They might be only digital, but a student who can see the progress he makes might be much more interested in continuing a course. A teacher who can see the influence he has, who can see how students learn and where problems are might be much better able to improve his content and be motivated to do so.

Also, scientists should not have to worry about presenting their studies. A lot of people know how to create graphics, some people know better about (the English) language and others are experts in LaTeX. If people were able to create requests online, get rewards for helping others and provide rewards to show that they really need help, I guess much better research could be made. Of course, this should be open.

For every unit / paper, there should be definitions what is necessary to know. The topics should automatically be linked to content that provides the knowledge.

It is very complex to plan such a system, because education in different languages / nations might be very different and ideas how to educate vary a lot. Even subtasks (creating a LaTeX editor, a graph that shows influence of papers / books by citation, creating an image editor, creating a reward system) are very difficult. And everything has to scale for millions of users. This means you would have to plan quite a lot before you could even think about implementation. For this project, you would need:

Somebody, who has experience with online courses.
A teacher for children.
A teacher for teenagers.
A teacher for students.
Somebody, who has experience with gamification.
Somebody, who has contacts to politics and knows how to advertise.

Wikipedia AI

Try to categorize images in Category:Uncategorized images or find images that have the wrong category / missing categories.

Translations

I did some translation work some years ago for LordsAWar. From a software point of view, this was a pain in the ass. Rosetta (part of Launchpad, for Ubuntu) is much better.

Another idea that is much better is Duolingo.

getlocalization.com might also be worth a try.

Wiki-like Dictionary

Wikipedia is great, but the Wiktionaries suck. I would like to have a dictionary service. It should be working for all language combinations (English ↔ German; English ↔ French; German ↔ French; …). The data should initially be filled by computers, but then be improved / corrected by humans.

Database

It is basically a database with a nice interface.

The database should have the following tables:

Languages: LangID, Name in the language itself, icon
Literature: LiteratureID, LangID, ISBN
Words: WordID, Word, Normalized Occurences in Standard Literature
WordPronounciation: WordPronounciationID, Pronounciation in phonetics, Pronounciation by a human
- Pronounciation by a human should be stored on Wikipedia Commons
WordTags: WordTagID, Tag, Description in Markdown
- Examples: Adjective, Substantive, male, genitiv, past, medicine …
TagTag: TagTagID, TagID, TagTag, Description in Markdown
- Examples: Gender, Word-Class, Tense, Context, …
Tags2Words: T2WID, TagID, WordID
Sentences: SentenceID, Sentence
Words2Sentece: ID, SentenceID, WordID
Definitions: ID, WordID, Definition, Image
- Images should be stored on Wikipedia Commons
Translations: ID, WordID, TranslationID
- Note that translations don’t have to be unique. There might be more than one correct translation for a word (e.g. “Bank”)
- Note that some translations might be more appropriate, depending on the context.
Users: UserID, DisplayName, E-mail, HashedPassword

Initial Data

Wiktionaries
For nouns: Lemmas of articles

Features

Downloadable minimal dataset for language combinations (e.g. on your smartphone in case you don’t have internet access). The most important words (2000 or so) should come with the audio data.
Web search like dict.leo.org (e.g. example search)
Forum to ask for translations, given context.
Discussion pages for entries
Moderators to “protect” entries.
Ranking to find most important words which need some human work

Distributed, Universal Tagging System

Tagging

Most information can be displayed rather simple. A string that describes the kind of information and a bool / int / float / string / BLOB for the information itself and an identifier.

For example, you can describe a product with the following labels:

Identifier: “898a4c822ffc456fa7a417e500b2c05a”
“898a4c822ffc456fa7a417e500b2c05a”, “ISBN-10”: “0141439513” (string)
“898a4c822ffc456fa7a417e500b2c05a”, “ISBN-13”: “978-0141439518” (string)
“898a4c822ffc456fa7a417e500b2c05a”, “Pages”: 480 (int)
“898a4c822ffc456fa7a417e500b2c05a”, “Publisher”: “Penguin Classics” (string)
“898a4c822ffc456fa7a417e500b2c05a”, “Category”: “Book” (string)
“898a4c822ffc456fa7a417e500b2c05a”, “Category”: “Literature” (string)
“898a4c822ffc456fa7a417e500b2c05a”, “Category”: ed054753e4b240a8aa1322ad348bf728 (identifier)
“898a4c822ffc456fa7a417e500b2c05a”, “VIEW”: 0839c5beac414fb19c400b6ca0372388 (identifier)
Identifier: “ed054753e4b240a8aa1322ad348bf728”
“ed054753e4b240a8aa1322ad348bf728”, “Name”: “Literature”
“ed054753e4b240a8aa1322ad348bf728”, “Category”: “Books”

As you can see, it is possible to create nested categories with this structure. You an also create lists this way.

Now clients should store information like this and share it.

Views

When information is presented like this, it is quite useless. But what about this kind of presentation:

Pride and prejudice infobox (defined here)

Or, for example ark.intel.com:

Intel Arc compare processors

So another required feature of such a client are “views”. A view is defined by an identifier (so that you can tag views just like any other object) and an HTML template. Objects could have labels called “VIEW” with type identifier that tell the client which view should be added.

Distribution

There are plenty of cool tools out there (Amazon recommendations, ark.intel.com to compare Intel processors, blackberry allows you to compare their phones, Wikipedia info boxes, …). But most of them are very ristricted. For example, the way I compare smartphones is not fundamentally different from the way I compare processors. Yes, the attributes differ. But basically it is creating a table with all the information. Also, Intel does not provide information about AMD processors.

So we need a way to get and share information. XML is the way-to-go for centralized computer systems. Maybe they can also be used to realize what I’m thinking about. But I think a problem that has to be solved is that we don’t have a single source for all information that we trust in. We have networks of trust. When Intel says A and a friend says B about an Intel processor, I guess I will rather believe A. But when Intel does not provide some information about a processor and a friend says B, but a person I don’t know says C, I’ll believe B. But when thousands of people say C and my friend says B, I might rather believe C.

It’s getting complicated, right? Maybe the processor example is not good, as there is much information and information is either right or wrong. But lets say we talk about genre of movies. This might be much more difficult as there is no “definitely right” or “definitely wrong”. Multiple answers might be right.

So every information has also have to carry information about who thinks it is right. And you have to be able to define networks you trust in. Perhaps you could create “people objects” that can also be labeled. “Your” object had to be protected so that only you could add “friend of” labels or “I trust” labels or something like this.

Python Code Search

Do you know Debian Code Search? Michael Stapelberg, the creator of it, described how he did it in his bachelor’s thesis.

I would like to have the same for Python code. This could be done by downloading all Python packages. I already did this part, see Analyzing PyPI Metadata and the follow-up post which is still on my TODO-list to be published (see draft).

It would also be possible to add GitHub repositories.

Comment.it

Sometimes, I just want to add my 2ct to something. I would like to have a central website / service - preferable as a browser plugin - with which I can add comments to other websites on an URL basis (including anchors for maximum exactness). Then I could read what other people think of something, even if there is no comments section.

Charity Search & Find

“Vermittlung” (recruiting?) of people who have the ability needed for charity organizations.

People:

profile / abilities
- Text, grade / level?
- tags?
location
Karma?
Filtering

Projects:

Profile page for each project
photos
members / oranization / roles / karma / contributers
domain

Other elements:

Forum: phpBB? Disqus?
File uploads: Imgur?
Calendar
E-Mail list: GNU Mailman?
Surveys: Limemonkey?

Database administration interface

It would be nice to have a database administration interface similar to phpMyAdmin for other databases, too (postgres, sqlite). One could also make it a web service (dbadmin.io or something similar).

Chrome

I had some ideas how to improve Chrome.

MATE

After the changes in desktop environments, MATE got my favorite desktop environment. Although I was more happy with GNOME 2.6.

Adding the drag-and-drop effect that creates a new window from an tab, known from Chrome, to Terminal and Pluma (gEdit).
Creating a LaTeX plugin for Pluma that auto-completes the environments.

LaTeX Tools

I would like to create online tools (pure HTML/CSS/JavaScript) that make the following tasks simpler.

A table editor. I know Trubens table tool, but this tool does not allow to combine cells. Also, the site is down quite often.
A TikZ editor.
An editor for bibliography.
A LaTeX source code beautifier.
A LaTeX-aware spell checker. This spell checker could probably use aspell, but it would have to filter LaTeX code.

Also, I would like to create an app that helps users to create formulas. This app should run on smartphones and on tablets. I don’t think that this can be done with pure JavaScript.

Open Hardware

I’m fascinated by the idea of open hardware. That means that you publish plans of something and maybe also how to create it. Although I don’t have any experience in this field, I can think of some interesting projects. One way to support open hardware would be to create an education and science platform, like the one I’ve described above.

Open Internet

I guess most smartphone users know this situation: You go to a friend / on vacation and you don’t have WLAN. This means you have to use mobile internet, which is expensive. If you’re in an area where not many people live, it is ok. If you’re in a big city, it is not. There are so many people who have an internet connection and a router which already establishes a WLAN. You can see them, but not use the connection! What a shame!

What we would need is a device with the following attributes:

Simplicity
- It has to be a DSL modem and a router combined, eventually also a DSL filter.
- Everything has to be configurable via web interface. This interface has to be VERY GOOD.
- You should be able to get a backup file via web interface that contains every single configuration. This file should be an good documented XML file. The documentation should contain example data.
- Every setting should have its own url, just like in Google Chrome.
- As many self-tests that give meaningful messages as possible:
  - A LED that indicates if the device has power.
  - Ethernet jack should glow if a device is connected and blink if data is send.
  - A software test via web interface that checks if internet connection is available.
  - Direct feedback when you enter wrong / malformed credentials.
- A reset button that restores the software completely from non-erasable memory.
- Small memory and rechargeable battery that allows you to download router software updates when the battery is full.
- A user manual with pictures that explains what to do to get internet.
Functionality and requirements
- It has to be able to create a WLAN.
- It has to be fast. I think currently 802.11n is with 450 Mbit/s the best you can get for WLAN and 1000 Mbit/s for Ethernet
- At least one Ethernet jack.
- It should be secure (WPA2, eventually don't support WEP and WPA).
- Reasonable energy consumption and no active fans.
- A standardized power supply unit that can be bought without buying a new device.
Box - how it looks
- The case should be robust.
- You should be able to mount it to a wall or to lay it on the floor.

I don’t think it is necessary to support VoIP, ISDN and Surf Sticks.

Now the special part: It should allow you to create a WLAN that others can use by registering in a service. The device should guarantee that you get the bandwidth, in case you need it. But if you have free bandwidth, others should be able to use it. Of course, this function should also protect you from legal trouble. An essential problem is keeping you from legal trouble while making sure that nobody uses the system to betray external users. But when you solve this problem, I guess it would be quite easy to establish free WLAN in all bigger cities. A great chance for tourism and a backup-option for you when your internet connection breaks.

The service should also allow the user to register the free WLAN online. An app should download these locations and be able to navigate a user to the next free WLAN.

Ah and of course everything in there should be free. This piece of hardware is critical for your internet access. If you want to be sure that you don’t get under surveillance by an attack on this piece of hardware, it would be good to know that some smart people had the possibility to check if everything is fine with this hardware.

Work computers

Today, we have a lot of computers that are used for very, very simple work. The most computing intensive part might be large Excel sheets. So basically, they don’t need any improvements in hardware for years. But the few things they do, need to be done well. Security is important. It is also important that things are stable and don’t change a lot. And what they do should be fast. Loading times are almost not acceptable.

I guess many tasks could be done within a browser. So work that needs heavy computation can be done on a stronger machine (the cloud - not necessarily outside of the company).

Why hasn’t any big company like General Motors, General Electric, Wallmart or even countries that have thousands of schools and government employees tried to create such a computer that is really reliable, robust and cheap (energy and because it can be produced it can be produced in very big numbers)?

Here is what I think should be ok:

processor with low power consumption (700 MHz or more)
2 GB of RAM (I guess you might now think of this missatributed Bill Gates quote … but with cat /proc/meminfo you can see how much you currently use).
30 GB SSD: Important information should be stored on a computer that is protected very well against data loss. A SSD is silent and can read content very fast. Ideally, only the OS is stored on the employees computer.
VERY silent fan, if possible non at all.
Big monitor with high resolution, because those people have to work all day with the computer and low quality speakers.
Good and silent keyboards (like the CODE keyboard).
Network card.
Graphic card that allows the high resolution display.

This is just a quick thought. I think such a system should contain some reference software that has to run fluidly. The software should also be open, of course. I think the following should be enough:

Linux based OS (e.g. Debian)
Basic command line tools (bash, grep, find, cat, vim)
Desktop manager with classic desktop metaphor (e.g. MATE)
File manager with access to a network drive (e.g. Caja or Nautilus)
Modern Browser (Firefox or Chrome)

Tasks that can (and should) be done via browser are:

E-mails: e.g. Roundcube
Excel: Hmmm … I know that Google Docs offers some similar stuff. Bug I guess it can’t replace Microsoft Excel by now. I don’t know if there are any self-hosted services
Word: e.g. Etherpad
Outlook: e.g. OwnCloud
LaTeX: e.g. FlyLaTeX
Geographic information systems: I don’t know if there is software online. But I guess with OpenStreetMaps it should not be too difficult to create it. ArcGis seems to be one solution.

Basically, you can do almost everything with a web application. So the client can get quite slim. But although you could probably do everything with a self hosted client/server solution, those solutions don’t always exist yet.

Tasks that should not be done via browser might be:

Professional video/audio editing: I guess you need more than one monitor to display all relevant information.
Programming: Although I have seen Cloud9, I doubt that programming in the cloud can be convenient in the next years. How does bug fixing work? How about manual testing? Whats with parallel execution?
Messaging: If you want to use encrypted communication (e.g. E-mail with PGP) you should probably do the encryption on your machine.

Hmmm … astonishingly, I can currently not think of more tasks.

Not so smart phone

Do you remember the good old days when your cell phone wasn’t essentially a small PC? I’ve bought a smartphone a while ago (article), but I still see reasons to have a cell phone:

Battery life: My Motorola W156 had a battery with only 940 mAh, but 465 hours stand-by time. If it had 3100 mAh as the Samsung Galaxy Note II, it would have a standby time of 1534 hours! That are about 64 days!
Security: Have you ever heard of somebody hacking a device that can only phone and send SMS?
Cost: The Motorola W156 costs 25 Euro on Amazon.
Robustness: A friend of mine put her Nokia 3310 accidentally in the washing machine. After that, she removed the battery, let it dry for a week, put the battery back. It worked. What the hell!?! (See also: Nokia is forever and Indestructible Nokia 3310 meme)
Size: Modern smartphones are a little bit uncomfortable to phone with. They are too big, although they are very thin. A size of 114 x 43 x 14 mm is fine, maybe a little bigger is also ok.

The needed functionality is:

Phone with good quality
Send SMS (and repetedly try to do so if no net is available)
Store about 100 contacts
Save about 100 SMS
Load battery via micro USB (Common External Power Supply)
If not too complicated: Let me back up all data on the phone via this micro-USB slot and let me also restore such a backup
3.5mm phone jack for using a headset

</ul>

What is not needed:

Camera, Flashlight
Internet access, Bluethooth, NFC, …
Multi-colored display: B/W screen is just ok
Fingerprint scanner

Light Alarm Clock

An alarm clock which wakes you up with light.

Features:

Set multiple alarms
(Non-lit) display which shows
- the time
- time to the next alarm in X d - Y h - Z m format
Batteries
Be able to set multiple alarms:
- Choose days for which the alarm rings
- Choose dimming profile for the alarm (smooth - aprubt)
Be able to change the (dimming) light which is inside

Bitcoin

info@martin-thoma.de (Martin Thoma) — Wed, 11 Sep 2013 20:53:51 +0200

Bitcoin [...] is a cryptographic currency where the creation and transfer of bitcoins is based on an open-source cryptographic protocol that is independent of any central authority. Bitcoins can be transferred through a computer or smartphone without an intermediate financial institution. The concept was introduced in a 2008 paper by a pseudonymous developer known only as "Satoshi Nakamoto", who called it a peer-to-peer, electronic cash system. The processing of Bitcoin transactions is secured by servers called bitcoin miners. These servers communicate over an internet-based network and confirm transactions by adding them to a ledger which is updated and archived periodically using peer-to-peer filesharing technology. In addition to archiving transactions, each new ledger update creates some newly minted bitcoins. The number of new bitcoins created in each update is halved every 4 years until the year 2140 when this number will round down to zero. At that time no more bitcoins will be added into circulation and the total number of bitcoins will have reached a maximum of 21 million bitcoins. To accommodate this limit, each bitcoin is subdivided down to eight decimal places; forming 100 million smaller units called satoshis. In August 2013 Germany's Finance Ministry subsumed Bitcoins under the term "unit of account"—a financial instrument—though not as e-money or a functional currency. Although bitcoin is promoted as a digital currency, many commentators have criticized bitcoin's volatile exchange rate, relatively inflexible supply, high risk of loss, and minimal use in trade.

Source: Wikipedia

Sounds interesting, doesn’t it? I’d like to share my thoughts about some aspects of Bitcoin. Please note that I’m neither an economics student, nor did I read the specifications of Bitcoin.

Security of Bitcoin

Data loss

Once in a while, hard discs crash. If you’re unlucky, you will lose all your data. This could mean you lose your Bitcoin wallet, so you lose your money. There is absolutely no way to get the money back. It’s lost. Just like if you’ve lost bank notes in a big city. You will never get it back (source).

BUT, unlike with bank notes, nobody else will get it. These Bitcoins are lost and nobody can do anything against that. I think that means after 2140 the number of active Bitcons will drop! This is called a deflation with normal money. But as you can subdivide every bitcoin in as small pieces as you like, I’m not sure what this means for economy.

Attacks

When money is involved, there will always be people who try to profit from that. So you have to think about security of Bitcoins.

System attacks

For me, a system attack is an attack where somebody tries to break the system. He does NOT want to profit from the system, but he wants to harm all people who use it.

The only attack I’ve found is a 51%-attack (source). This means, in long term you have to have more than 50% of the computing power of the total bitcoin network. So if there is a dramatic change in computing power, this might make Bitcoin as a system insecure. Also, in short term with much luck you might also take down the network with less computing power (source).

But the more people participate, the more unlikely it gets. And this problem might our current system also have. If somebody manages to get MUCH more computing power, he could break encryption. This means, all bank transactions could at least be read by this attacker, if not manipulated.

Theft

Theft is possible by stealing your credentials. This can be done by infecting your computer with a virus, just like it can be done today with your bank account if you use online banking (or with the computers of banks, though that should be much more difficult).

Also, the software you use for Bitcoin transactions might have errors.

Money laundering

It seems to be a big problem of Bitcoin that money laundering might be easy. You can’t close a criminals account, you cannot stop criminals using Bitcoin for illegal operations.

But what can you do with the current currency against that? Not much, when the money gets abroad.

Credits

You can’t give credits as banks do it today with Bitcoin. If I understand it correctly, the “Common Equity Capital Ratio” has to be at least 3.5% in 2013 (source). In other words: banks are allowed to give more than 27x the amount of money they actually have! Now I’m very uncertain if I’m here correct, but I think this was an important reason why the financial crisis happened. This would not have been possible with Bitcoin! With Bitcoin, you can only give credit for money you currently have. No cent … uhmm … Sotisho more than that.

BUT: I really don’t know if this would be better. I guess it would be extremely difficult for states to get money. There would be the need of heavy restructuring in how we finance our countries.

Interest

A problem with credits in the Bitcoin-system would be that it is very difficult to estimate what the result of the interest would be.

As there are “only” 21 million Bitcoins, it would bet possible that the sum of all dept is more than available Bitcoins. Can it be legal to force people to pay when you know that at least one technically can’t pay at the end? This problem seems not to exist in the current system, as they always print new money.

Wages

At the moment, employed people have to fight in unions for more money. They really need it, because inflation happens all the time. So just to keep the current situation, employees have to do something.

With Bitcoin, the situation might change. As we produce more and more every year, a deflation would be likely. This means, bosses would have to cut loans to keep the situation the same. But it is much more difficult to change the bills you pay. People will not blindly accept less money on their accounts per month. Unlike today, when people are basically ok when they get a compensation for inflation.

How to apply the Viterbi algorithm

info@martin-thoma.de (Martin Thoma) — Wed, 11 Sep 2013 17:46:22 +0200

The goal of the Viterbi algorithm is find the most likely sequence of hidden states given some observed events.

Lets say this is your HMM:

A hidden Markov model (HMM) example

We always start in $x$ and always end in $z$.

Now you observed the sequence $O_1 = BAB$. What is the most likely sequence that would generate this path?

Candidates are:

xxyz
xyyz
xyzz

If you’re learing this because you will write the exam at KIT, you might have such an diagram:

Scheme of the Viterbi algorithm

In this case, the bold path is the Viterbi path. You can see this when you get backwards from the last state:

$\frac{1}{125} \cdot 1 \cdot 0.2 > \frac{9}{2500} \cdot 0.8 \cdot 0.2$. After this step, you only have one choice. So the Viterbi path is xyzz.

Please note that there might be multiple paths with the highest possibility.

As every state depends only on the state before, you can get the most likely path step by step. In every step, you calculate how likely it is that you end up in state x, state y, state z. After that, it doesn’t matter how you got there. So you always have to expand those three nodes. You will get 9 following states, but only three of them matter (for each resulting state, the paths that had the highest probability leading there).

How do I calculate a Histogram equalization?

info@martin-thoma.de (Martin Thoma) — Wed, 11 Sep 2013 13:17:39 +0200

Let’s say you have the following greyscale image:

$A = \begin{pmatrix} 255 & 50 & 255\\ 0 & 50 & 50 \end{pmatrix}$

Histogram

Now the histogram is a function $H: [0,255] \rightarrow \mathbb{N}_0$ .

The histogram of $A$ is

$H(x) := \begin{cases} 1 &\text{, if } x = 0\\ 3 &\text{, if } x = 50\\ 2 &\text{, if } x = 255 \end{cases}$

Accumulated histogram

The accumulated histogram $H_\alpha: [0,255] \rightarrow \mathbb{N}_0$ is defined as

$H_\alpha(x) := \sum_{i=0}^x H(i)$

This means, in the given example you get

$H_\alpha(x) := \begin{cases} 1 &\text{, if } x < 50\\ 4 &\text{, if } 50 \leq x < 255\\ 6 &\text{, if } x = 255 \end{cases}$

Normalized histogram

The normalized histogram is defined as $H_n(x) := \mathrm{round}(\frac{255}{w \cdot h} \cdot H_\alpha(x))$ where $w$ is the width of the image and $h$ is the height of the image.

In our example it’s:

$H_n(x) := \begin{cases} 43 &\text{, if } x < 50\\ 170 &\text{, if } 50 \leq x < 255\\ 255 &\text{, if } x = 255 \end{cases}$

So the resulting image is

$A = \begin{pmatrix} 255 & 170 & 255\\ 43 & 170 & 170 \end{pmatrix}$

Calculations with quaternions

info@martin-thoma.de (Martin Thoma) — Mon, 09 Sep 2013 14:45:17 +0200

Quaternions are an expansion of the concept of complex numbers on structures with four (instead of two) components. A quaterion $h$ can be written as a vector or in the form of $h = h_0 + ih_1 + j h_2 + kh_3$ , where $i, j$ and $k$ are related to the $i$ in complex numbers. Accordingly $h_0$ is often called real part and h_1, h_2, h_3 are called imaginary part of a quaternion.

For $i, j$ and $k$ the following rules are applied:

$i^2 = j^2 = k^2 = -1$ and $ijk=-1$

Basic rules

From these rules follows: $\begin{align} ij &= k\\ ji &= -k\\ jk &= i\\ kj &= -i\\ ki &= j\\ ik &= -j \end{align}$

(proof)

Multiplication

Now, obviously quaternions multiplication is not commutative: $ij = k \neq -k = ji$

But for numbers in $\mathbb{R}$ , it is commutative (proof).

The multiplication is:

$\begin{align} x y &=( x_0 y_0 - x_1 y_1 - x_2 y_2 - x_3 y_3)\\ &+( x_0 y_1 + x_1 y_0 + x_2 y_3 - x_3 y_2) \mathrm i\\ &+( x_0 y_2 - x_1 y_3 + x_2 y_0 + x_3 y_1) \mathrm j\\ &+( x_0 y_3 + x_1 y_2 - x_2 y_1 + x_3 y_0) \mathrm k \end{align}$

Calculating multiplicative inverse

This means, when you’re given an element $x = x_0 + x_1 \mathrm i + x_2 \mathrm j + x_3 \mathrm k$ its inverse $y$ can be calculated by solving the following linear system of equations:

$\begin{align} x_0 y_0 - x_1 y_1 - x_2 y_2 - x_3 y_3 &= 1\\ x_0 y_1 + x_1 y_0 + x_2 y_3 - x_3 y_2 &= 0\\ x_0 y_2 - x_1 y_3 + x_2 y_0 + x_3 y_1 &= 0\\ x_0 y_3 + x_1 y_2 - x_2 y_1 + x_3 y_0 &= 0\\ \end{align}$

which can be written as:

$\left(\begin{array}{cccc|c} x_0 & -x_1 & -x_2 & -x_3 & 1\\ x_1 & x_0 & -x_3 & x_2 & 0\\ x_2 & x_3 & x_0 & -x_1 & 0\\ x_3 & -x_2 & x_1 & x_0 & 0 \end{array}\right).$

According to mathworks it is

$y = \frac{x_0 - \mathrm i x_1 - \mathrm j x_2 - \mathrm k x_3}{x_0^2 + x_1^2 + x_2^2 + x_3^2}$

Conjugate

Just like the complex conjugate is defined as

$\overline{z} = a - ib$ with $z=a+ib$

the conjugate of quaternions is defined as

$\bar x := x_0-x_1\mathrm i-x_2\mathrm j-x_3\mathrm k$ with $x=x_0+x_1\mathrm i+x_2\mathrm j+x_3\mathrm k$

Rotating points

Quaternions can be used to rotate points. It works like this:

$x' = q x \overline{q}$

Pretty simple, isn’t it?

For example, if you had the point (2,0,0) and you wanted to rotated it by $q = (\frac{\sqrt{2}}{2}, 0, \frac{\sqrt{2}}{2}, 0)$ you would transform (2,0,0) to (2i+0k+0j) and calculate

$\begin{align} x' &= q x \overline{q}\\ &= (\frac{\sqrt{2}}{2}+\frac{\sqrt{2}}{2} \mathrm j) \cdot 2 \mathrm i \cdot (\frac{\sqrt{2}}{2} - \frac{\sqrt{2}}{2} \mathrm j)\\ &= (\sqrt{2} \mathrm i + \sqrt{2} \mathrm{ji}) \cdot (\frac{\sqrt{2}}{2} - \frac{\sqrt{2}}{2} \mathrm j)\\ &= (\sqrt{2} \mathrm i - \sqrt{2} \mathrm k) \cdot (\frac{\sqrt{2}}{2} - \frac{\sqrt{2}}{2} \mathrm j)\\ &= \mathrm i - \mathrm{ij} - \mathrm k + \mathrm{kj}\\ &= \mathrm{i - k -k - i}\\ &= -2 \mathrm k\\ &\Rightarrow x' = (0,0,-2) \end{align}$

Rotation matrix to quaternion

Let $M$ be a rotation matrix and $m_{ij}$ be an entry of this matrix.

General rule

\begin{align} s &= \frac{1}{2} \sqrt{1 + \sum_{i=1}^3 m_{ii}}
x &= \frac{m_{32} - m_{23}}{4s}
y &= \frac{m_{13} - m_{31}}{4s}
z &= \frac{m_{21} - m_{12}}{4s} \end{align}

resulting in the quaternion $(s, x, y, z)$ .

Special rule

A rotation matrix

$R_y = \begin{pmatrix} \cos(\theta) & 0 & \sin(\theta)\\ 0 & 1 & 0\\ -\sin(\theta) & 0 & \cos(\theta) \end{pmatrix}$

can be transformed to a quaternion

$q = (\cos(\frac{\theta}{2}), \vec u \sin (\frac{\theta}{2}))$

where $\vec u$ describes the axis you rotate by.

In this case $R_y$ is the y-axis, so $\vec u = \begin{pmatrix}0\\1\\0\end{pmatrix}$ .

Informatik am KIT

info@martin-thoma.de (Martin Thoma) — Sat, 07 Sep 2013 11:41:05 +0200

Ich werde immer wieder gefragt, wie Informatik am KIT ist. Was kann ich über Karlsruhe erzählen? Wieviel Praxis bekommt man in einem Informatik-Studium am KIT?

Leider denke ich nicht, das ich wirklich gut darüber schreiben dann, da ich nur „Informatik I“ an der Uni Augsburg gehört habe. Sonst war ich nur am KIT. Auch über Karlsruhe kann ich nicht so viel erzählen, wie man sich vielleicht denkt. Ich bin meistens am studieren. Daher habe ich von Karlsruhe außerhalb der Innenstadt bisher wenig gesehen.

Ein paar Anhaltspunkte könnte die Wikipediaseite des KIT sowie die einstieg-informatik.de-Seite geben.

Genug allgemeines bla-bla. Nun lasse ich die Disclaimer-Sätze (meinem Eindruck nach, ich kenne eigentlich nur das KIT und habe keine aussagekräftigen Vergleiche, …) weg und erzähle etwas über Karlsruhe und das KIT.

Das Studium

Allgemeines

Die Regelstudienzeit des Bachelor-Informatik-Studium beträgt 6 Semester und 180 ECTS. Davon sind 15 ECTS für die Bachelor-Arbeit, 6 ECTS für Softskills (z.B. Sprachkurse und verpflichtent 2 ECTS für Teamarbeit in der Softwareentwicklung) und folgende Plicht-Kurse vorgesehen:

Lehrveranstaltung	ECTS	Weitere Links
1. Semester
Grundbegriffe der Informatik	4
Programmieren	5	Mein Tutorium
Höhere Mathematik I	9
Lineare Algebra I	9
2. Semester
Algorithmen I	6
Softwaretechnik I	6
Rechnerorganisation	6
Höhere Mathematik II	6
Lineare Algebra II	5
3. Semester
Theoretische Grundlagen der Informatik	6
Praxis der Software-Entwicklung (PSE)	6
Betriebssysteme	6
Digitaltechnik und Entwurfsverfahren	6
Wahrscheinlichkeitstheorie und Statistik	4.5
4. Semester
Datenbanksysteme	4
Rechnernetze	4
Numerik	4.5
5. Semester
Algorithmen II	6
Programmierparadigmen	6
6. Semester
Bachelorarbeit	15
$\sum$	124

Ich habe „Analysis I+II“ anstelle von „Höhere Mathematik I+II“ und „Lineare Algebra und analytische Geometrie I+II“ anstelle von „Lineare Algebra I+II“ gehört. Das sind die Mathematiker-Kurse.

Wie ihr nun erkannt haben werdet, bleiben $180 - 124 - 6 = 50$ übrig. Da ich die Mathematiker-Kurse gemacht habe (die jeweils 9 ECTS anstelle von 5 und 4 geben), bei mir sogar nur 41 ECTS. Davon muss man zwei Stammmodule für jeweils 6 ECTS machen. Dann sind es nur noch $50-2 \cdot 6 = 38$ ECTS.

Die Praxis der Softwareentwicklung (PSE) ist übrigens eine tolle Sache am KIT-Informatik-Studium. Dort sucht man sich aus einer vorgegebenen Auswahl an Projekten eines aus. Dieses bearbeitet man in einer Gruppe von 5 Studenten für ein halbes Jahr. Ich habe an einem Projekt des Frauenhofer Instituts gearbeitet, das sich UpToDatE (Upload automation and data entry) nannte. Mal schauen, vielleicht schreibe ich mal etwas darüber.

Stammmodule

Computergraphik
Echtzeitsysteme
Formale Systeme
Kognitive Systeme
Rechnerstrukturen
Sicherheit
Softwaretechnik II
Telematik

Proseminare

Die Informatik-Proseminare sind leider sehr schlecht organisiert. Ich habe auf GitHub ein öffentliches Repository, wo ihr hoffentlich aktuelle Informationen über Info-Proseminare finden (bzw. auch eintragen) könnt.

Softstkills

Von den 6 ECTS für Softskills sind ja schon 2 verpflichtend in Teamarbeit für die Softwareentwicklung (TSE - gehört zu PSE). Das ist nur ein Kniff mit dem Modulhandbuch, weil man wohl kein Stammmodul mit 8 ECTS machen konnte / wollte, also hat man PSE mit 6 ECTS und 2 ECTS für TSE. Praktisch bedeutet PSE auch so viel Arbeit, dass 8 ECTS angebracht sind (eher sogar mehr) und man muss im Team arbeiten. Man bekommt die Punkte also nicht geschenkt, aber man muss auch nichts außerhalb von PSE dafür machen.

Von den 4 übrigen Punkten kann man sich aus vielen interessanten Kursen des HOC etwas aussuchen, z.B. Kurse des Sprachenzentrums. Diese sind deutlich besser als die Sprachkurse in der Schule. Ich habe mich für English C1: Advanced entschieden. Da ich aber auch ein Tutorium gehalten habe (was 4 ECTS und 8,67 Euro/Stunde gibt), hätte ich das nicht machen müssen. Der Kurs war aber so toll, dass ich es nicht bereue.

Weitere Kurse

Ich habe folgendes belegt:

Web-Engineering	4
Algorithmen für planare Graphen	5

SEHR viele weitere Kurse sind im Modulhandbuch zu finden.

Hier ist übrigens mal ein Abhängigkeitsgraph zum KIT-Bachelor-Informatik:

Abhängigkeiten im Bachelor-Informatik Studium am KIT

Es ist am KIT als Informatiker übrigens sehr leicht eine HiWi-Stelle zu finden.

Ergänzungsfächer

Man kann am KIT aus einer Fülle von ~~Nebenfächern~~ Ergänzungsfächern wählen:

Elektro- und Informtionstechnik
Maschinenbau
Mathematik
Physik
Grundlagen des Rechts
Volkswirtschaftslehre
Betriebswirtschaftslehre
Operation Research

Auf Anfrage kann man aber wohl noch weitere machen. So studiert ein Bekannter in Richtung Medizinische Bildgebungsverfahren einiges.

Mathematik

Proseminar	3	Mein Vortrag zum Proseminar Diskrete Mathematik
Einführung in die Algebra und Zahlentheorie	9
Topologie und Geometrie	9

Man kann noch Analysis III und Funktionalanalysis / Hilberträume machen und vermutlich geht noch viel mehr, wenn man nachfragt.

Sonstiges

Das KIT hat eine 24h-Bibliothek (ja, die ist wirklich 24h offen, auch Sonntags. Meines Wissens macht die nur an Weihnachten zu), die ATIS (tolle Lernräume, viele Computer, Drucker, ein Scanner), ein Rechenzentrum.

Der Campus ist zusammenhängend. Das, was als „Campus Nord“ bezeichnet wird ist ein Forschungsbereich. Dort werden keine Vorlesungen gehalten. Für Bachelor-Studenten ist der Campus Nord also irrelevant, außer ihr wollt da einen Job.

Tipps zum Studium

Ganz unbescheiden: Mein Blog ist toll ☺ Nein, im ernst, wenn du etwas vom Mathe-Stoff nicht verstehst, kann ich dir nur empfehlen mal rein zu schauen und vielleicht die Tags „linear algebra“ oder „analysis“ zu durchstöbern. Ich blogge zu relativ vielen Themen, mal auf Deutsch und manchmal auf Englisch, die meistens für einen KIT-Informatik-Studenten von Interesse sind. Eventuell ist auch meine kleine Linkliste interessant.
Mathe ist interessant, aber wohl nicht jedermanns Sache. Ich denke ein guter Informatiker sollte auch gute Mathe-Kenntnisse haben. Noch kann ich nicht beurteilen, ob sich für mich das Besuchen der Mathe-Kurse (+Klausuren) gelohnt hat. Es war auf jeden Fall deutlich schwerer. Aber ich bin davon überzeugt, dass es sich lohnen wird.
Bleib am Ball: Gerade in Mathe - egal ob Analysis oder HM - muss man immer versuchen am Ball zu bleiben. Vorbereitung (das Kapitel im Skript lesen) und Nachbereitung (am Abend kurz aufschreiben: Was wurde gemacht? Und dann kontrollieren: Kann ich das auch?) lohnen sich. Bzw., ich habe das im 2. Semester nicht gemacht und es hat sich bei der Klausurvorbereitung gerächt.
Die ersten Vorlesungen sind besonders wichtig. In fast allen gibt es ein gutes Skript, das auch online ist. Fast überall wird genau das Skript gemacht, nicht mehr und nicht weniger. Aber manchmal werden Dinge im Skript eher betont oder vielleicht etwas genauer erklärt/ein Beispiel mehr gemacht. Also: Skript direkt am Anfang ausdrucken und immer mitnehmen.
Drucken: Bis auf die Skripte würde ich alles in der ATIS (und nicht im SCC) drucken. Du solltest nach der O-Phase beide Accounts haben.
Übungsblätter: Es lohnt sich, die Übungsblätter ordentlich zu machen. Sie sind, neben Altklausuren, die beste Klausurvorbereitung. Also: Mache deine Lösungen kleinschrittig, eventuell mit Erklärungen die du in einem Semester nutzen kannst, schreibe sie sauber auf und mach alle. Egal wie schwer sie sind oder wie wenig Zeit du hast. Das wird - im 2. Semester - anstrengend, aber es lohnt sich. Also nicht nur auf die Punkte schauen, die du für den Übungsschein brauchst, sondern auch an die bevorstehende Klausur denken.

FAQ

Frage: Ich kann nich programmieren. Ist das schlimm? Antwort: Nein! Am KIT lernt man alles von Grund auf. In „Programmieren“ lernt man Java, in „Betriebssysteme“ grundlegendes C, in „PSE“ - je nach dem was man macht - C++, Java, C#, in „SWT I“ grundlagen über paralleles Programmieren mit Java.
Frage: Wie Praxisnah ist das Studium? Antwort: Kommt darauf an, was man später machen will und was man als Vergleich nimmt. Vermutlich sind die Fachhochschulen näher an der Praxis, wenn man auf reines Programmieren hinaus will. Aber wenn ihr einfach nur im Beruf programmieren wollt, müsst ihr nicht studieren. Vielleicht macht dann eine Ausbildung mehr Sinn.
Frage: Was lernt man in der Informatik, wenn nicht Programmieren? Antwort: Vieles. Algorithmen und Datenstrukturen, wie genau ein Rechner von einzelnen Transistoren, über Schaltungen, Schaltwerke, Maschinencode und Assembler funktioniert, was theoretische Berechnungsmodelle sind, was überhaupt berechenbar ist, wie Computer lernen können, wie man mit Ungenauigkeit in Berechnungen umgeht, wie man Programme/Probleme sinnvoll strukturiert, was Sicherheit in der Informatik ist und wie man verschiedene Sicherheitsbegriffe zeigen kann...
Frage: Was ist charakteristisch für die Informatik am KIT? Antwort: Das KIT hat eine große Informatik-Fakultät. Wenn man hier Informatik zu studieren beginnt, kann man sich in alle (oder zumindest viele) Richtungen weiterentwickeln. Außerdem ist das Informatik-Studium am KIT sehr Mathematik-lastig.
Frage: Ich würde gerne XYZ lernen, aber das steht nicht im Modulhandbuch. Kann ich das trotzdem am KIT lernen? Antwort: Es gibt einen E-Mail-Verteiler für Studenten. Dort kann man einfach mal fragen, ob andere auch interesse daran haben. Außerdem gibt es Hochschulgruppen. Bei über 3000 Studenten wird sich schon jemand finden, der auch XYZ machen will / es kann.

Karlsruhe

Am KIT werden immer wieder interessante Vorträge gehalten, z.B.

Google Tech Talk: JavaScript and V8 - A functional-ish language and implementation in the mainstream
TalKIT
Weitere

Es gibt den Science Slam Karlsruhe, Physik am Samstag und sehr viele weitere Veranstaltungen.

Falls man mehr über Karlsruhe wissen will: Auch dazu gibt es einen Wikipedia-Artikel.

Wohnungen

Tja, eine Wohnung zu finden ist in Karlsruhe schwer. Mit der Wohnungssuche sollte man sehr früh beginnen. Es gibt zwar einige Wohnheime, aber auch die sind voll. Und die Qualität der Wohnheime ist sehr unterschiedlich. Ich habe gehört, dass das HaDiKo und die Insterburg nicht so toll sind, aber das an der Waldhornstraße soll sehr gut sein. Es gibt neben den Wohnheimen in dieser Liste noch mindestens eine weiteres (das evangelische Studentinnen-Wohnheim in Rüpurr).

Die zentralen Wohnungen sind meistens Altbauwohnungen. Ich habe meine über studenten-wg.de gefunden.

Allgemein gilt jedoch: Umso früher man sucht, umso besser.

Für einen Eindruck von Karlsruhe kann man sich meine Photo-Spheres anschauen.

Gedanken zur Bundestagswahl 2013

info@martin-thoma.de (Martin Thoma) — Fri, 06 Sep 2013 11:54:58 +0200

Am 22. September 2013 findet die Bundestagswahl 2013 statt.

Worum geht es?

Parteien

Ich werde jeweils zwei Videos verlinken. Eines von „MrWissen2go“ und einen Wahlwerbespot der Parteien selbst. Die Parteien sind aufsteigend nach vermutlichem Ergebnis geordnet.

Piraten


Mitgliederzahl	31.669
Durchschnittsalter	38,9 Jahre
Frauenanteil	ca. 5–15 Prozent
Themen	Transparenz Bedingungsloses Grundeinkommen Datenschutz und Internet
Wahlprogramm	Vollständig als PDF (recht groß) als Hörbuch

DIE LINKE.


Mitgliederzahl	63.761
Durchschnittsalter	60 Jahre
Frauenanteil	37,7 Prozent
Themen	10 Euro Mindeslohn Höhere Steuern auf hohe Einkommen Höhere Sozialleistungen
Wahlprogramm	Vollständig als PDF (912 kB) als Hörbuch (6.1 MB)

Ich konnte leider keinen “offiziellen” Wahlwerbespot finden. Aber der folgende Clip kommt dem am nächsten:

FDP


Mitgliederzahl	58.675
Durchschnittsalter	53 Jahre
Frauenanteil	23,0 Prozent
Themen	Staatsschulden abbauen ohne Steuererhöhung Bürgergeld Soli abschaffen
Wahlprogramm	Vollständig als PDF

Grüne


Mitgliederzahl	60.808
Durchschnittsalter	48 Jahre
Frauenanteil	37,8 Prozent
Themen	Höhere Steuern auf hohe Einkommen Kostenlose Kita-Plätze 8,50 Euro Mindestlohn Bürgerversicherung für alle Massentierhaltung abschaffen Gentechnik verhindern ÖPNV billiger machen
Wahlprogramm	Vollständig als PDF Hörbuch

SPD


Mitgliederzahl	472.469
Durchschnittsalter	59 Jahre
Frauenanteil	31,5 Prozent
Themen	Höhere Steuern für höhere Einkommen Wahlalter auf 16 Jahre senken Kostenlose Kita-Plätze Volle Rente ab 63 Jahren für alle, die min. 45 Jahre gearbeitet haben
Wahlprogramm	Vollständig als PDF

CDU / CSU


Mitgliederzahl	469.575
Durchschnittsalter	59 Jahre
Frauenanteil	25,6 Prozent
Themen	konservativ (im Großen und Ganzen soll sich nichts ändern) mehr Kameraüberwachung Mieten sollen im öffentlichen Raum nicht weiter ansteigen
Wahlprogramm	Vollständig als PDF (683.47 kB) als Hörbuch (1.44 MB)

Koalitionsmöglichkeiten

Koalitionsrechner - Ergebnis

Die Parteien haben viele aussagen über Koalitionen nach der Bundestagswahl gemacht:

Für CDU/CSU kommt die FDP und die SPD in Frage. (Quelle)
Die FDP will nur mit der Union eine Koalition. (Quelle)
Rot-Rot-Grün wird es nicht geben. (Quelle, siehe auch im Wahlprogramm der Grünen: „Die Linke steht abseits des grünen Wandels.“)
Die SPD will eigentlich nur mit den Grünen. (Quelle)

Nun kann man Umfrageergebnisse betrachten:

Umfrageergebnisse zur Bundestagswahl 2013
Quelle: wahlrecht.de

Damit steht fest:

CDU / CSU bleibt in der Regierung.
Wenn die FDP die 5%-Hürde schafft, gibt es wieder Schwarz-Gelb.
Sonst wirds vielleicht interessant, aber vermutlich Schwarz-Rot

Themen

Ich finde es interessant, welche Themen für die Parteien wichtig sind und welche in der Öffentlichkeit häufig diskutiert werden. Im folgenden mal einige Themen, die meiner Meinung nach wichtig sind.

Sozialer Ausgleich

Ich denke, dass in Deutschland die viel zitierte „Schere zwischen Arm und Reich“ zu groß wird. Zu dem Thema gibt es in den Wiki-Artikeln Einkommensverteilung in Deutschland und Vermögensverteilung in Deutschland viele Informationen, z.B.:

2008 betrug nach Zahlen des DIW das mittlere verfügbare Einkommen 1.252 Euro
die ärmsten 50% haben 1,4% des Vermögens in Deutschland
die Top 0,1 % haben 22,5% des Vermögens in Deutschland
die Top 10% haben 66,6% des Vermögens in Deutschland
5 Mio. verdienen weniger als 8,50 Euro pro Stunde (Quelle)
Im Jahr 2011 sind an mehr als 1,21 Millionen Bedarfsgemeinschaften mit Aufstockern durchschnittlich 737 Euro je Monat gezahlt worden. (Quelle)

Es ist sehr einfach, was dagegen hilft:

Höhere Steuern auf höhere Einkommen
Mindestlohn
Höhere Sozialleistungen für Arme
Höhe Erbschaftsvermögen (um eine konkrete Zahl zu nennen: > 10,000,000 Euro) müssen stark mit Erbschaftssteuer eingeschränkt werden. Davor darf es keine „Steuerflucht“ geben. (Existiert dieses Problem?)

Ich bin außerdem davon überzeugt, dass viele Menschen, die eigentlich Anspruch auf Sozialleistungen hätten, diesen nicht in Anspruch nehmen. Zum einen, weil sie es nicht wollen (Quelle), zum anderen aber vermutlich auch, weil sie nicht wissen, dass sie Anspruch haben.

Hier mal ein paar Leistungen, die es gibt:

In der Kategorie:Sozialleistung (Deutschland) findet man noch viel mehr.

Eurokrise

Die Eurokrise muss beendet werden. Wichtige Zahlen dazu sind:

Die Jugendarbeitslosigkeit in Spanien betrug 2012 ca. 24,9%.
Griechenland: Arbeitslosigkeit bei Menschen mit Hochschulbildung unter 24 Jahren: 28,8% - in Deutschland 12,8%
In Griechenland betrug die Arbeitslosigkeit 2011 ca. 17,6%
Die Staatverschuldung ist katastrophal - egal ob man Deutschland, Frankreich, Spanien, Griechenland oder Italien anschaut.

Hier kann man nicht davon ausgehen, dass irgendeine Partei eine belegbar gute Lösung vorbringt. Wie auch? Das Problem gab es noch nie. Aber die Frage nach dem, was man zu tun gedenkt ist doch interessant:

CDU / CSU:
- Für: Sparmaßnahmen, Schuldenbremse, Stabilitätspakt, Europäische Bankenunion
- Gegen: Eurobonds
SPD:
- Für: Finanztransaktionssteuer
DIE LINKE.:
- Für: Einmalige Abgabe von 10% auf Vermögen zur Schuldentilgung bei einem Freibetrag von 1.000.000 Euro auf Privatvermögen und 2.000.000 Euro auf Betriebsvermögen; Finanztransaktionssteuer
Piraten:
- Für: Finanztransaktionssteuer; Schuldenschnitte
Grüne:
- Für: Vermögensabgabe (vergleichbar mit LINKE)

Korruption und Transparenz

Abstimmung Schärfere Regeln gegen Abgeordnetenbestechung

Energiewende

Offshore Windkraftparks

Netzausbau

Erneuerbare-Energien-Gesetz

Einspeisevergütung

Steuern

Konzerne spielen Länder gegeneinander aus
Steuern sind in Deutschland kompliziert

Quelle

Erbschaftssteuer: Sie sorgt dafür, dass sich Reichtum weniger stark anhäuft.
Umsatzsteuer: Sie stellt einen einfachen Weg dar, wie der Staat an Geld kommt. Sie betrifft alle gleichmäßig. Ich finde es nicht sinnvoll, hier zwischen verschiedenen Klassen (Lebensmittel, Luxusgüter) zu unterscheiden. Wenn Lebensmittel mehr kosten würden, würde man ihren Wert vielleicht mehr schätzen.
Einkommensteuer: Auch mit der Lohnsteuer bekommt der Staat viel Geld. Aber im Gegensatz zur Umsatzsteuer kann man hier Leute, die viel Verdienen stärker belasten.
Energiesteuern: Durch Energiesteuern kann man Anreize schaffen, sparsame Geräte / Infrastruktur zu entwickeln. Dies sollte für die Bürger auf lange Sicht von Vorteil sein.
Rauschmittelsteuern: Eigentlich sollten Menschen nicht rauchen und saufen. Also sollten diese Produkte - obwohl sie eventuell billig in der Herstellung sind - teuer verkauft werden. Die Steuer sollte süchtigen dazu dienen, von ihrer Sucht frei zu kommen bzw. in die Suchtprävention gesteckt werden.
Grundsteuer: Durch eine hohe Grundsteuer nutzt vermutlich derjenige, der am meisten damit verdienen kann, das Grundstück. Sie kann also eine gute Verteilung dieser knappen Ressource ermöglichen.

Solidaritätszuschlag
Kirchensteuer: Das sollte die Kirche selbst machen!
Arbeitslosenversicherung: Sollte durch Umsatz- und Einkommenssteuer gedeckt werden. Grundstätzlich denke ich sollte hier jeder gleich versichert sein. Wenn jemand mehr als grundlegende Leistungen will, soll er das Privat machen. Aber eine gewisse Grundsicherung soll jeder bekommen.
Krankenversicherung: Siehe Arbeitslosenversicherung
Pflegeversicherung: Siehe Arbeitslosenversicherung
Tabak, Alkopop, Schaumwein: Sollte eine „Rauschmittelsteuer“ sein
Grunderwerbssteuer: Warum wird Grunderwerb extra gezählt? Warum nicht einfach nur Umsatzsteuer?
Kapitalertragsteuer: Warum ist das nicht einfach Einkommenssteuer?
Versicherungsteuer, Kraftfahrzeugsteuer: Siehe Grunderwerbsteuer
Stromsteuer: Inwiefern unterscheided sie sich von der Energiesteuer?

Am Ende eines Jahres wird das Einkommen (Lohn, Kapitalerträge) berechnet.
Sei dieses Gesamteinkommen `$x$` (in Euro):
- `$x \leq 12.000$`: Keine Steuer
- `$x \leq 24.000$`: 15% Steuer. Die zu bezahlende Steuer ist also `$(x - 12.000) \cdot \frac{15}{100}$`
- `$x \leq 36.000$`: 20% Steuer. Die zu bezahlende Steuer ist also `$(36.000 - 12.000) \cdot \frac{15}{100} + (x-24.000) \cdot \frac{20}{100}$`
- ...
- `$x > 120.000$`: 50% Steuer. Die zu bezahlende Steuer ist also `$(36.000 - 12.000) \cdot \frac{15}{100} + (36.000-24.000) \cdot \frac{20}{100} + \dots + (x-120.000) \cdot \frac{50}{100}$`

Steueraufkommen in Deutschland (Stand: 2008)

Flat Tax

Bürgerrechte

Spähskandal

Verkauf von Meldedaten, manchmal auch „Bestandsdatenauskunft“ genannt (Quelle)
Mehr Videoüberwachung
Friedrich nimmt NSA in Schutz
Vorratsdatenspeicherung (siehe Abstimmung, EU-Richtlinie)

Quelle

Kleine Themen

Wahl-o-mat

Adoptionsrecht / Gleichstellung für Homosexuelle
Betreuungsgeld
Syrien-Krise
Generelles Tempolimit auf Autobahnen
NATO-Austritt Deutschlands; Rüstungsexporte; Militärische Beteiligungen Deutschlands
Aufnahme von mehr Flüchtigen
Einwanderung
Lohnersatzleistungen bei Pflege von Angehörigen
Frauenquote in Aufsichtsräten
Renteneintrittsalter
Ehegattensplitting
Autobahn-Maut für Ausländer

Abschließende Worte

Bin ich mit der Politik der letzten 4 Jahre zufrieden?
Was ist mir wichtig?

Wenn man Grundsätzlich mit der Politik der letzten Jahre zufrieden ist, sollte man CDU/CSU wählen.
Wenn man Wert auf Transparenz legt, und will das Internetkompetenz in das Parlament kommt, sollte man die Piraten wählen.
Wenn man auf Umweltschutz, neue (aber nicht radikale) Ideen, wert legt, sollte man die Grünen wählen
Wenn man extreme Änderungen im sozialen Bereich will, sollte man DIE LINKE wählen.
Wenn man CDU/CSU und FDP wieder sehen will, und will das auf Datenschutz-Aspekte mehr wert gelegt wird, sollte man FDP wählen.
SPD: Hmm ... wenn man das gleiche in Rot will, was man schon kennt? Ich kann keinen wirklichen Themenschwerpunkt finden, in dem sich die SPD stark von der CDU/CSU abhebt.

Gauß'sche Zahlen und verwandte Ringe

info@martin-thoma.de (Martin Thoma) — Wed, 04 Sep 2013 09:16:41 +0200

Sei $\mathbb{Z}[\sqrt{z}]$ mit $z \in \mathbb{Z}$ der kleinste Ring, der $\mathbb{Z}$ und $\sqrt{z}$ enthält. Sei $A := \{a + b \sqrt{z} | a, b \in \mathbb{Z}\}$ .

Behauptung: $\mathbb{Z}[\sqrt{z}] = A$ Beweis: z.Z.: $\mathbb{Z}[\sqrt{z}] \subseteq A$ und $\mathbb{Z}[\sqrt{z}] \supseteq A$

Zuerst zeige ich $\mathbb{Z}[\sqrt{z}] \supseteq A$ :

`$\mathbb{Z}[\sqrt{z}]$` enthält `$\mathbb{Z} \Rightarrow \{a \in \mathbb{Z}\} \subseteq \mathbb{Z}[\sqrt{z}]$`
`$\mathbb{Z}[\sqrt{z}]$` enthält `$\sqrt{z} \Rightarrow \{\sqrt{z}\} \subseteq \mathbb{Z}[\sqrt{z}]$`
`$\mathbb{Z}[\sqrt{z}]$` ist ein Ring, also ist `$(\mathbb{Z}[\sqrt{z}], +)$` eine abelsche Gruppe.
`$\Rightarrow A \subseteq \mathbb{Z}[\sqrt{z}]$`

Nun $\mathbb{Z}[\sqrt{z}] \subseteq A$ :

`$(A, +) \leq (\mathbb{R}, +)$`, denn (UG-Kriterium)
- `$0 = 0 + 0 \sqrt{z} \in A \Rightarrow A \neq \emptyset$`
- `$\forall (a+b \sqrt{z}), (c + d \sqrt{z}) \in A:$` `$(a+b\sqrt{z}) - (c+d\sqrt{z}) = \underbrace{(a-c)}_{\in \mathbb{Z}} + \underbrace{(b-d)}_{\in \mathbb{Z}} \sqrt{z} \in A$`
`$(A, +)$` ist abelsch, da `$(\mathbb{R}, +)$` abelsch ist
`$(A, \cdot)$` ist Untermonoid von `$(\mathbb{R}, \cdot)$`, denn
- `$1 = 1 + 0 \sqrt{z} \in A$`
- `$\forall (a+b \sqrt{z}), (c + d \sqrt{z}) \in A:$` `$(a+b\sqrt{z}) \cdot (c+ d\sqrt{z}) = \underbrace{(ac + z bd)}_{\in \mathbb{Z}} + \underbrace{(ad + bc)}_{\in \mathbb{Z}} \sqrt{z} \in A$`
Distributivgesetze: Vererben sich aus `$(\mathbb{R}, \cdot)$`
`$\Rightarrow \mathbb{Z}[\sqrt{z}] \subseteq A$`

$\Rightarrow \mathbb{Z}[\sqrt{z}] = A \blacksquare$

How to solve linear congruence equations

info@martin-thoma.de (Martin Thoma) — Sun, 01 Sep 2013 17:43:49 +0200

When you have a system of linear congruences like:

$$ \begin{align} x &\equiv 4 \mod 19\\ x &\equiv 12 \mod 37\\ x &\equiv 14 \mod 43 \end{align} $$

you can solve it quite easily. Johannes Schickling has written a very nice JavaScript Application that applies the following algorithm online. I’ve used his source code to write the following Python code.

Pseudocode

Solve a system of linear congruences

Python

#!/usr/bin/env python
# -*- coding: utf-8 -*-

def ExtendedEuclideanAlgorithm(a, b):
	"""
		Calculates gcd(a,b) and a linear combination such that
		gcd(a,b) = a*x + b*y

		As a side effect:
		If gcd(a,b) = 1 = a*x + b*y
		Then x is multiplicative inverse of a modulo b.
	"""
	aO, bO = a, b

	x=lasty=0
	y=lastx=1
	while (b!=0):
		q= a/b
		a, b = b, a%b
		x, lastx = lastx-q*x, x
		y, lasty = lasty-q*y, y

	return {
		"x": lastx,
		"y": lasty,
		"gcd": aO * lastx + bO * lasty
	}

def solveLinearCongruenceEquations(rests, modulos):
	"""
	Solve a system of linear congruences.

	>>> solveLinearCongruenceEquations([4, 12, 14], [19, 37, 43])
	{'congruence class': 22804, 'modulo': 30229}
	"""
	assert len(rests) == len(modulos)
	x = 0
	M = reduce(lambda x, y: x*y, modulos)

	for mi, resti in zip(modulos, rests):
		Mi = M / mi
		s = ExtendedEuclideanAlgorithm(Mi, mi)["x"]
		e = s * Mi
		x += resti * e
	return {"congruence class": ((x % M) + M) % M, "modulo": M}

if __name__ == "__main__":
	import doctest
	doctest.testmod()

How to calculate the Legendre symbol

info@martin-thoma.de (Martin Thoma) — Sun, 01 Sep 2013 13:05:46 +0200

Pseudocode

Pseudocode: Calculate Legendre symbol

Python

#!/usr/bin/env python
# -*- coding: utf-8 -*-

def isPrime(a):
    return all(a % i for i in xrange(2, a))

# http://stackoverflow.com/a/14793082/562769
def factorize(n):
    factors = []

    p = 2
    while True:
        while(n % p == 0 and n > 0): #while we can divide by smaller number, do so
            factors.append(p)
            n = n / p
        p += 1  #p is not necessary prime, but n%p == 0 only for prime numbers
        if p > n / p:
            break
    if n > 1:
        factors.append(n)
    return factors

def calculateLegendre(a, p):
	"""
	Calculate the legendre symbol (a, p) with p is prime.
	The result is either -1, 0 or 1

	>>> calculateLegendre(3, 29)
	-1
	>>> calculateLegendre(111, 41) # Beispiel aus dem Skript, S. 114
	-1
	>>> calculateLegendre(113, 41) # Beispiel aus dem Skript, S. 114
	1
	>>> calculateLegendre(2, 31)
	1
	>>> calculateLegendre(5, 31)
	1
	>>> calculateLegendre(150, 1009) # http://math.stackexchange.com/q/221223/6876
	1
	>>> calculateLegendre(25, 1009) # http://math.stackexchange.com/q/221223/6876
	1
	>>> calculateLegendre(2, 1009) # http://math.stackexchange.com/q/221223/6876
	1
	>>> calculateLegendre(3, 1009) # http://math.stackexchange.com/q/221223/6876
	1
	"""
	if a >= p or a < 0:
		return calculateLegendre(a % p, p)
	elif a == 0 or a == 1:
		return a
	elif a == 2:
		if p%8 == 1 or p%8 == 7:
			return 1
		else:
			return -1
	elif a == p-1:
		if p%4 == 1:
			return 1
		else:
			return -1
	elif not isPrime(a):
		factors = factorize(a)
		product = 1
		for pi in factors:
			product *= calculateLegendre(pi, p)
		return product
	else:
		if ((p-1)/2)%2==0 or ((a-1)/2)%2==0:
			return calculateLegendre(p, a)
		else:
			return (-1)*calculateLegendre(p, a)

if __name__ == "__main__":
	import doctest
	doctest.testmod()

How many Homomorphisms exist between Z/nZ and Z/mZ?

info@martin-thoma.de (Martin Thoma) — Tue, 27 Aug 2013 15:26:38 +0200

Today I’ve wondered how many homomorphisms are between the groups $(\mathbb{Z}/n\mathbb{Z},+)$ and $(\mathbb{Z}/m\mathbb{Z},+)$ with $m, n \geq 2$ . Does it make a difference if I use + or $\cdot$ as operators?

Let $M := (\mathbb{Z}/m\mathbb{Z},+)$ and $N := (\mathbb{Z}/n\mathbb{Z},+)$ be groups and $\varphi: M \rightarrow N$ be a group homomorphism. Let $H := \{\varphi: M \rightarrow N\}$

Some fundamental theorems (which I’m not going to prove) are:

`$\varphi(M)$` is a group.
`$K_\varphi := \{m \in M | \varphi(m) = e_N\}$` is a group.
Lagrange's theorem: `$H \leq G \Rightarrow \#H | \# G$`
`$\varphi(1)$` defines the complete homomorphism as `$\varphi(n) = \varphi(1 + (n-1)) = \varphi(1) + \varphi(n-1)$`
`$\varphi(1) = 0$` is always a homomorphism (that maps everything to 0).

`$m = n$`

Theorem: $n=m \Rightarrow |H|=n$ Proof: You can map $1$ to $n$ values $\stackrel{(IV)}{\Rightarrow}$ there can’t be more than $n$ homomorphisms.

For every $i \in \{0, \dots, n-1\}$ exists an homomorphism $\varphi_i(1) = i$ : $ \begin{align} \varphi(a) + \varphi(b) &= (ai \mod n) + (bi \mod n)\\ &= ai + bi \mod n\\ &= (a+b)i \mod n\\ &= \varphi(a+b) \end{align} $

This means, that all of them are actually homomorphisms. For different $i,j \in \{0, \dots, n-1\}$ the homomorphisms $\varphi_i$ and $\varphi_j$ are different. So we really have $n$ homomorphisms $\blacksquare$

My first thought was that it’s only a permutation, but

`$m \in M$`	0	1	2	3	4	5
`$\varphi(m) \in N$`	0	3	0	3	0	3

is not a permutation, but a homomorphism.

Distinct primes

Theorem: $n, m \in \mathbb{P}, n \neq m \Rightarrow |H| = 1$ Proof:

Let $n, m \in \mathbb{P}, n \neq m$ .

Part 1: $1 \leq |H|$

This follows directly from (V).

Part 2: $|H| \leq 1$

We know that $\begin{align} \stackrel{(I)+(III)}{\Rightarrow} &\# \varphi(M) | \# N\\ \stackrel{n \in \mathbb{P}}{\Rightarrow} &\# \varphi(M) \in \{1, n\} \end{align}$

and

$\begin{align} \stackrel{(II)+(III)}{\Rightarrow} &\# K_\varphi | \# M\\ \stackrel{m \in \mathbb{P}}{\Rightarrow} &\# K_\varphi \in \{1, m\} \end{align}$

Case 1: $m > n$

Now the kernel can’t be trivial anymore, so $\# K_\varphi = m$ . This means that everything is mapped to 0. There is only one homomorphism that does so.

Case 2: $m < n$

Now the image $\varphi(M)$ can’t be all of $N$ . This means $\varphi(M) = 1$ which is again the 0-mapping $\blacksquare$

Any `$n$` and `$m$`

Theorem: $|H| = gcd(n, m)$ Proof:

First a sanity check: The theorems above are special cases of this theorem. Let’s try to prove it.

Let $n$ be composed of primes $p_1, \dots, p_x$ (where $p_i = p_j$ is allowed). Then $N = \mathbb{Z}/n\mathbb{Z} \cong \mathbb{Z}/p_1\mathbb{Z} \times \mathbb{Z}/p_2\mathbb{Z} \times \cdots \times \mathbb{Z}/p_x\mathbb{Z}$ according to the Chinese remainder theorem. The same is true for $M$ . As there are $p_i$ homomorphisms between $\mathbb{Z}/p_i\mathbb{Z}$ and $\mathbb{Z}/p_j\mathbb{Z}$ with $p_i = p_j$ and as you can take a $p_i$ from the left and a $p_j$ from the right you can combine the different homomorphisms. So it is basically a combinatoric problem. As everything (except for same primes) will only have 1 homomorphism, you have to multiply the number of homomophisms for each pair $(p_i, p_j)$ . But this is simply the gcd $\blacksquare$

`$(\mathbb{Z}/n\mathbb{Z}, \cdot)$`

What changes when we use $(\mathbb{Z}/n\mathbb{Z}, \cdot)$ and $(\mathbb{Z}/m\mathbb{Z}, \cdot)$ ?

Well, first of all $m$ and $n$ have to be primes. Otherwise, not every element would have an inverse. Second, you have to exclude 0. Which means we only use units:

$(\mathbb{Z}/n\mathbb{Z}, \cdot)^\times$ and $(\mathbb{Z}/m\mathbb{Z}, \cdot)^\times$

According to the German Wikipedia (source):

`$(\mathbb{Z}/n\mathbb{Z})^\times$` is cyclic `$:\Leftrightarrow n \in \{p^r, 2p^r | p \in \mathbb{P}, r \in \mathbb{N}_{\geq 1}\}$`

So there is no simple way to reduce it to the same proofs. But I’ve created a little script that automatically finds homomorphisms.

Quick way to find the number of the group homomorphisms ϕ:Z3→Z6?

Why is the stabilizer subgroup a subgroup?

info@martin-thoma.de (Martin Thoma) — Mon, 26 Aug 2013 11:24:54 +0200

Let `$(G, \cdot)$` be a group and `$M$` a set. A group action is a function: `$G \circ M \rightarrow M$` that satisfies the following two conditions

Identity: `$\forall m \in M: e_G \circ m = m$`
Associativity: `$\forall g, h \in G, m \in M: (g \cdot h) \circ m= g \circ (h \circ m)$`

Let `$m \in M$`. Then `$G_m := \{g \in G | g \circ m = m\}$` is called the stabilizer of `$m$`.

Theorem: $G_m$ is a group

Proof:

Let $m \in M$ .

$\stackrel{identity}{\Rightarrow}e_G \in G_m$

Let $a \in G_m$ . This means, that $a \circ m = m$ . And $\exists a^{-1} \in G$ , as $G$ is a group. $a^{-1} \cdot a = e_G$ , this means $(a^{-1} \cdot a) \circ m = m$ . $\stackrel{associativity}{\Rightarrow}a^{-1} \circ (a \circ m) = m \Leftrightarrow a^{-1} \circ m = m \Leftrightarrow a^{-1} \in G_m$

Let $a, b \in G_m$ . Then: $a \circ m = m$ and $b \circ m = m$

$\Rightarrow a \circ (b \circ m) = m$ $\stackrel{associativity}{\Rightarrow} (a \cdot b) \circ m = m \Leftrightarrow (a \cdot b) \in G_m \blacksquare$

Why is the intersection of two normal subgroups a normal subgroup?

info@martin-thoma.de (Martin Thoma) — Sun, 25 Aug 2013 18:04:56 +0200

Let $(G, \cdot)$ be a group and $X \lhd G$ and $Y \lhd G$ be two normal subgroups.

I will show this in two steps:

Show that `$X \cap Y$` is a group
Show that `$X \cap Y$` is a normal group of `$(G, \cdot)$`

Intersection of two subgroups is a subgroup

Theorem: $(X \cap Y) \leq G$ Proof:

$X \cap Y$ is not empty: $e_G \in X \land e_G \in Y \Rightarrow e_G \in (X \cap Y)$

$X \cap Y$ has inverse elements. Let $a \in (X \cap Y)$ . As a is in $X$ and $X$ is a group, $a^{-1} \in X$ . The same is true for $Y$ . So: $\forall a \in (X \cap Y) \exists a^{-1} \in (X \cap Y): a \cdot a^{-1} = a^{-1} \cdot a = e_G$

$\forall a,b \in (X \cap Y): a \cdot b^{-1} \in (X \cap Y)$ , because both, $a$ and $b^{-1}$ are in $X$ . As $X$ is a group, the result has to be in $X$ . Same argumentation for $Y$ . Then the result is in $X$ and $Y \blacksquare$

Intersection of two normal subgroups is normal

First the definition of a normal subgroup:

Let `$N \leq G$` be a subgroup of `$G$`. `$N \lhd G :\Leftrightarrow \forall n \in N \forall g \in G: g \cdot n \cdot g^{-1} \in N$`

Theorem: $(X \cap Y) \lhd G$

Proof:

$X \cap Y$ is a subgroup of $G$ as I have proved above.

$\forall n \in (X \cap Y) \forall g \in G: g \cdot n \cdot g^{-1} \in X$ and $\forall n \in (X \cap Y) \forall g \in G: g \cdot n \cdot g^{-1} \in Y$

$\Rightarrow \forall n \in (X \cap Y) \forall g \in G: g \cdot n \cdot g^{-1} \in (X \cap Y)$

$\Rightarrow (X \cap Y) \lhd G \blacksquare$

How to analyze Mailman archives

info@martin-thoma.de (Martin Thoma) — Sun, 18 Aug 2013 21:23:49 +0200

All mailing lists I use are GNU Mailman lists. This software provides archives of all Emails that were send over the list.

They look like this:

GNU Mailman list archive

Once in a while, I would like to search if a topic was already discussed. Here is how you can do it:

Download archives

wget --save-cookies cookie.txt --post-data 'username=user&password=pass' -A gz -m -p -E -k -K -np https://lists.abc.org/mailman/blub/

Rename archives

for file in *.txt.gz; do mv "`$file" "$`{file%.txt.gz}.txt"; done

To make them sortable:

for file in *January.txt; do mv "$file" "${file%January.txt}01.txt"; done
for file in *February.txt; do mv "$file" "${file%February.txt}02.txt"; done
for file in *March.txt; do mv "$file" "${file%March.txt}03.txt"; done
for file in *April.txt; do mv "$file" "${file%January.txt}04.txt"; done
for file in *May.txt; do mv "$file" "${file%May.txt}05.txt"; done
for file in *June.txt; do mv "$file" "${file%June.txt}06.txt"; done
for file in *July.txt; do mv "$file" "${file%July.txt}07.txt"; done
for file in *August.txt; do mv "$file" "${file%August.txt}08.txt"; done
for file in *September.txt; do mv "$file" "${file%September.txt}09.txt"; done
for file in *October.txt; do mv "$file" "${file%October.txt}10.txt"; done
for file in *November.txt; do mv "$file" "${file%November.txt}11.txt"; done
for file in *December.txt; do mv "$file" "${file%December.txt}12.txt"; done

Analyze them

To analyze the archives properly, you should perhaps first import all emails in a relational database. But with grep you could also do simple keyword searches.

Audible

info@martin-thoma.de (Martin Thoma) — Tue, 13 Aug 2013 00:58:21 +0200

I have just decided to try Audible.de, the German part of Audible.com which is “An Amazon Company”. They offer audiobooks. You have to know that I really like to listen to audiobooks when I can’t read and need some distraction.

Everybody who lives in Germany and buys something at Amazon knows the coupons that you get with a purchase:

Audible coupon - front
Audible coupon - back

Buying audiobooks

Buying the audiobook I liked (“Darknet” from Daniel Suarez) was just as I knew it from Amazon. Another design, but even my bank account data is there. After purchasing it, I wanted to listen to it:

After having bought an audiobook

After clicking on “my library” I get to this page:

My library

A click on “Start Audible-Assistant” redirects to:

Software download assistant

So I thought I could download it to my smartphone.

Android App

If the Audible Android App was software for your computer, you would call it spyware:

It requires the permission to add or change calendar dates,
send E-mails to "guests" (whatever this means),
pairing with Bluetooth devices,
and some other stuff (which seems to be okay)

Why the hell do they want to change my calendar dates? I only want to listen to audiobooks I’ve bought…

This is not okay for me, so lets see if I can listen to it on my computer.

Linux

As so often, there is no support for Linux. But Wine does a great job in bringing Windows applications to Linux. So I’ve installed the software:

Audible Manager

I thought, maybe I need to activate it via the web interface. Back to image “Software download assistant”, click on “Add device” (the big orange button):

Audible Web assistant

My E-mail

There seems to be no way for me to download an audiobook with Audible. As it’s now very late, I’ve contacted customer service and described the issue (12.08.2013, 23:00):

Sehr geehrte Damen und Herren, wenn ich mein Hörbuch herunterladen will, werde ich darauf hingewiesen, dass ich noch kein Gerät hinzugefügt habe. Wenn ich dann über den Link (http://www.audible.de/assistant) ein Gerät (meinen Linux-Computer auf dem der AudibleManager mit wine läuft) hinzufügen will, bekomme ich einen Fehler 404 (Seite nicht gefunden). Mit freundlichen Grüßen, Martin Thoma

Answer:

Lieber Herr Thoma, vielen Dank für Ihr freundliches Schreiben. Es tut uns leid, tatsächlich können wir unseren Service derzeit nicht auf Linux-Systemen anbieten. Ob es in Zukunft eine Unterstützung für Linux geben wird, können wir gegenwärtig leider nicht versprechen. Wir bitten um Ihr Verständnis. Wenn Sie den Manager bereits installiert haben und als Browser Firefox benutzen, dann sollten Sie das admhelper.adh Script mit dem AudibleDownloadHelper via wine öffnen können. Wir hoffen, dass dies eine gute Alternative für Sie ist und stehen bei allen weiteren Fragen gerne zur Verfügung. Ich wünsche Ihnen einen wunderbaren Tag. Herzliche Grüße [A name which I don't want to show here.] Audible-Kundenservice

Virtual Box

I’ve finally got it to work. I’ve installed XP with Virtual Box.

Cancellation

Step 0
Step 1
Step 2
Step 3
Step 4
Step 5
Step 6
Step 7

Conclusion

DRM seems to be necessary for such a service. Okay, I want to get my free audiobook, so I have to accept this. But this one reason why the service SUCKS. It would be much easier to let the user download the audiobook as a simple MP3 / WAV / OGG file. Everybody can play these files.

Do you want to install software for every service you want to purchase audiobooks / songs from? Please also consider that Amazon might close your account (source), so you can not simply stick with one provider.

Currently, I can only say: Don’t use audible. It’s a terrible service.

Improving lecture notes: Job (almost) done!

info@martin-thoma.de (Martin Thoma) — Sat, 10 Aug 2013 15:50:09 +0200

Some of you might know that I’ve bin improving the lecture notes for the computer engineering lecture (digital electronics) since April 2013.

How I've got the job

This was kind of funny. I send Prof. Dr. Asfour some notes of passages that could be improved (mainly typos). About two days later he proposed me to correct it by myself. Another day later I signed the contract. I’ve never signed a contract that fast.

What I did

My job was

to correct errors (German language, statements about computer science and LaTeX),
find parts that were outdated and update them,
find sections that were difficult to understand and simplify them and
to make it easier to make changes in the future (Well, I don't think Prof. Dr. Asfour thought this was my job ... but I think it's important.)

So Prof. Dr. Asfour created a SVN repository with all LaTeX sources of the latest lecture notes (which were already great!). He also sent me Emails he received from students who mentioned errors just like I did and some notes from a tutor who tried to improve the script some time ago.

Revisions

With svn checkout svn://somepath@1 working-directory you can checkout the first revision of a SVN repository.

Total number of files and folders: find . | wc -l

Revision 1: 1885
Revision 30: 1488

How often did I change files (source):

svn log -qvr 1:HEAD|perl -nle 'print if /^Changed paths:/ ... /^-+$/ and /^\s/' \
    | sort | uniq -c | sort -n

1    M /ti1.bib
      2    M /
      2    M /anhang-1.tex
      2    M /anhang-3.tex
      2    M /anhang-5.tex
      2    M /diss-report.cls
      2    M /figures/Makefile
      2    M /titel.tex
      2    M /zahlen_codes
      3    M /vorwort.tex
      7    M /einleitung.tex
      8    M /Makefile
     10    M /my_def.tex
     11    M /sw.tex
     14    M /arith.tex
     14    M /skript.tex
     16    M /daten.tex
     19    M /sn.tex
     22    M /README.txt
     28    M /skript.pdf

With CodeAnalyzer over all .tex files:

	Revision 1	Revision 30
Total files	31	14
Total Lines	24,679	11,077
Avg Line Length	39	45
Code Lines	18,500	8,967
Comment Lines	1,124	871
Whitespace Lines	5,177	1,391
Resulting PDF pages	229	233
Examples	93 <	93
Images	211	211

StackExchange

I’ve learned quite a lot about LaTeX while correcting the document. My questions on StackExchange might reflect that:

And some language questions:

What I've learned

You can open file skript.tex on line 1234 with vim +1234 skript.tex
grep -rniI, find, xargs, make and Meld are VERY useful
I like Git more than SVN (because I don't need internet to commit)
nag package is interesting
There seems to be no good way to create images for digital electronics which might contain LaTeX. xfig is the best I found, but it is very hard to use.

Conclusion

Working as a “Skript-HiWi” is easy work, but more time consuming than you might think. Even if you know how to work with LaTeX. Rebuilding a big document takes some time.

I was astonished that there were some topics which I didn’t understand yet. After I’ve prepared for the exam, I thought I knew everything in the lecture notes. Obviously, this was not the case (or I forgot how to do division with twos complement meanwhile).

Why everybody should know about conditional probability

info@martin-thoma.de (Martin Thoma) — Tue, 23 Jul 2013 15:37:39 +0200

Probability theory is difficult, but I think everybody should be taught basics in this subject. Why? Because it is relevant for everybody.

Suppose you go to the doctor to make a cancer test. The test is positive. That means you have cancer, right? Wrong!

Basics

A probability is a numerical value in [0, 1] that is assigned to events (or lets rather say outcomes of an experiment). A probability can never be less than zero and never be more than one.

The sum of all probabilities of all outcomes of one experiment is always 1.

Example: Your experiment $x$ could be throwing a coin. The outcome of your experiment is either head or tails. So if $Pr[x = \text{head}] = 0.6$ , then $Pr[x = \text{tail}] = 0.4$ .

Now you might argue that the coin can also stand on its border. This would result in a different model of the situation with different probabilities, e.g.:

$Pr[x = \text{head}] = 0.559$ , $Pr[x = \text{border}] = 0.001$ and $Pr[x = \text{tail}] = 0.4$ .

Another very important rule is Bayes rule:

$Pr[A|B] = \frac{Pr[B|A] \cdot Pr[A]}{Pr[B]}$

This works also with more variables:

$Pr[A|BC] = \frac{Pr[B|AC] \cdot Pr[A|C]}{Pr[B|C]}$

Cancer

Lets say the probability of having cancer is $Pr[C] = 0.01$ . This means the probability of not having cancer is $Pr[\neg C] = 0.99$ .

Now you have cancer tests. They can either be positive (+) which means they say you have cancer. Or they are negative (-), which means according to the test, you don’t have cancer.

They are not always working as expected. So you get so called “false positives” and “false negatives”.

A false positive is a positive result, while it should be negative. So the test says you have cancer, while you don’t have cancer. It is denoted by $Pr[+ | \neg C] = 0.2$ .

A false negative is a negative result, while it should be positive. So the test says you don’t have cancer, but you actually have cancer. It is denoted by $Pr[- | C] = 0.1$ .

You should note that $Pr[+ | \neg C] + Pr[- | C] = 0.2 + 0.1 = 0.3 \neq 1$ . Why is this the case? Because the test might also not be related at all to you having cancer.

But $Pr[+ | \neg C] + Pr[- | \neg C]$ has to be 1 and $Pr[+ | C] + Pr[- | C]$ also has to be one.

So you can draw this table:

`$Pr[\text{Testresult}\|\text{Cancer}]$`	`$C$`	`$\neg C$`
`$+$`	`$0.9$`	`$0.2$`
`$-$`	`$0.1$`	`$0.8$`

You can see that the correct results are much more likely than the wrong ones.

Some intermediate results

How likely is a positive / negative test result? $\begin{align} Pr[+] &= Pr[+|C] \cdot Pr[C] + Pr[+| \neg C] \cdot Pr[\neg C] = 0.207\\ Pr[-] &= Pr[-|C] \cdot Pr[C] + Pr[-| \neg C] \cdot Pr[\neg C] = 0.793 \end{align}$

How likely are the combinations? (This time you don’t know if you have cancer):

`$Pr[\text{Testresult}, \text{Cancer}]$`	`$C$`	`$\neg C$`
`$+$`	`$0.009$`	`$0.198$`
`$-$`	`$0.001$`	`$0.792$`

Reversing it

It is quite likely that you would like to know how likely it is that you have cancer. Without a test, you know:

$\begin{align} Pr[C] &= 0.01\\ Pr[\neg C] &= 0.99 \end{align}$

Now you get a positive test result. How likely is it that you have cancer? In other words: Calculate $Pr[C|+]$

$\begin{align} Pr[C|+] &= \frac{Pr[C, +]}{Pr[+]} = \frac{0.009}{0.207} = \frac{1}{23} \approx 0.043\\ Pr[\neg C|+] &= 1 - \frac{1}{23} = \frac{22}{23} \approx 0.957\\ Pr[C|-] &= \frac{0.001}{0.793} = \frac{1}{793} \approx 0.001\\ Pr[\neg C|-] &= 1 - \frac{1}{793} = \frac{792}{793} \approx 0.999 \end{align}$

How would you interpret these results? I’d say: When you get a positive result you shouldn’t really worry. But perhaps you should make other tests. When you get a negative results you can be very sure that you don’t have cancer.

Testing again

A very natural approach to a positive test result might be taking the same test again. How does this influence the probability?

There are four possibilities what could have happened:

You have cancer:
- Second test was negative
- Second test was positive
You don't have cancer:
- Second test was negative
- Second test was positive

First of all, I compare some intermediate results $ \begin{align} Pr[++] &= Pr[C] \cdot Pr[+|C] \cdot Pr[+|C] + Pr[\neg C] \cdot Pr[+|\neg C] \cdot Pr[+|\neg C]\\ &= 0.477 \neq 0.042849 = 0.207^2 = (Pr[+])^2\\ Pr[+-] &= Pr[C] \cdot Pr[+|C] \cdot Pr[-|C] + Pr[\neg C] \cdot Pr[+|\neg C] \cdot Pr[-|\neg C] \\ &= 0.01 \cdot 0.9 \cdot 0.1 + 0.99 \cdot 0.2 \cdot 0.8\\ &= 0.1593\\ Pr[C,+,+] &= Pr[C] \cdot Pr[+ | C]^2 = 0.01 \cdot 0.9^2 = 0.0081\\ Pr[C,+,-] &= Pr[C] \cdot Pr[+ | C] \cdot Pr[-|C] = 0.01 \cdot 0.9 \cdot 0.1 = 0.0009\\ Pr[C|++] &= \frac{Pr[C,+,+]}{Pr[++]} = \frac{0.0081}{0.207^2} = \frac{9}{530} \approx 0.170\\ Pr[C|+-] &= \frac{Pr[C,+,-]}{Pr[+-]} = \frac{0.0009}{0.1593} = \frac{1}{177} \approx 0.006 \end{align} $

Woooha! So if one test says you have cancer, don’t worry. When two tests say you have cancer, you have a 17% chance of having cancer.

On the other hand: One negative and one positive test is better than no test at all, because without a test you have a probability of 0.01 to have cancer. With both tests, you only have a probability of 0.006.

Why probability is important for you

As you might have seen, your intuition about probability is wrong. But we hear numbers all the time. It is important for us to understand them, so we should drop our intuition and learn how to use those numbers.

Kollisionsresistente Hashfunktionen und Einwegfunktionen

info@martin-thoma.de (Martin Thoma) — Tue, 23 Jul 2013 10:05:24 +0200

Definitionen

Sei `$f:X \rightarrow Y$` eine Funktion. `$f$` heißt eine Einwegfunktion, genau dann wenn für alle `$x \in X$` gilt:

`$y := f(x)$` kann in Polynomialzeit berechnet werden
Für die Berechnung eines Urbildes `$x$` aus `$y$` existiert kein randomisierter Algorithmus, der in Polynomialzeit läuft.

Eine Funktion `$H:\{0,1\}^* \rightarrow \{0,1\}^k$` heißt kollisionsresistente Hashfunktion, wenn gilt: Jeder effiziente Algorithmus findet nur mit kleiner Wahrscheinlichkeit eine Kollision.

Was heißt „kleine Wahrscheinlichkeit“? Nach dem Auswerten der Funktion $H$ für $x_1, x_2, \dots x_n$ sollte die Wahrscheinlichkeit nicht signifikant höher sein als $\displaystyle 1-\frac{n!\cdot{{2^k} \choose n}}{{2^k}^n}$ Diese Wahrscheinlichkeit kommt von dem Geburtstagsparadoxon bzw. dem Schubfachprinzip. Wir haben $2^k$ Schubfächer (Funktionswerte) in die wir die $x_i$ (Urbilder) einordnen können.

Satz

Behauptung: Jede kollisionsresistente Hashfunktion ist eine Einwegfunktion. Beweis: durch Widerspruch Sei $f$ eine kollisionsresistente Hashfunktion Annahme: $f$ ist keine Einwegfunktion

Dann existiert ein Angreifer $\mathcal{A}$ , der für eine Bild $f(x)$ ein $x'$ findet, sodass $f(x) = f(x')$ gilt.

Der Angreifer $\mathcal{B}$ macht nichts anderes, als zufällig Werte $x \in \{0,1\}^{2k}$ zu wählen, $f(x)$ zu berechnen, den Angreifer $\mathcal{A}$ auf $f(x)$ anzuwenden und zu überprüfen, ob das von $\mathcal{A}$ gelieferte $x' \neq x$ ist. Sobald das ein mal der Fall ist, hat der Angreifer gewonnen.

Nun wenden wir $f$ auf $x \in \{0,1\}^{2k}$ an. Es gilt:

$Pr_x[\underbrace{|f^{-1}(f(x))|}_{\substack{\text{Anzahl der Urbilder}\\\text{zum Hashwert} f(x)} } = 1] \leq \frac{2^k}{2^{2k}} = \frac{1}{2^k}$ .

Die $2^k$ im Zähler stehen für die Funktionswerte und die $2^{2k}$ für die Urbilder.

Nun ist $\frac{1}{2^k}$ eine vernachlässigbare Funktion.

$\Rightarrow$ Die Wahrscheinlichkeit, dass wir keine Kollsion finden ist vernachlässigbar.

$\Rightarrow$ Mit signifikanter Wahrscheinlichkeit hat $f(x)$ $k \geq 2$ Urbilder.

$\Rightarrow$ Die Wahrscheinlichkeit, dass $\mathcal{B}$ Kollisionen findet ist etwa $(1-\frac{1}{k}) \cdot m$ , wobei $m$ die Anzahl der Widerholungen ist.

Signs that it is too hot outside

info@martin-thoma.de (Martin Thoma) — Mon, 22 Jul 2013 22:59:13 +0200

Weather map
Wetterkarte vom 26.07.2013
Wetterkarte vom 04.07.2015

Simquadrat

info@martin-thoma.de (Martin Thoma) — Sun, 21 Jul 2013 18:18:12 +0200

Simquadrat ist ein Dienst von sipgate. Das besondere daran ist, dass Simkarten verkauft werden, die eine Festnetznummer haben. Ich habe das ganze ausprobiert und will meine Erfahrungen nun teilen.

Das Angebot

Man geht auf simquadrat.de, bezahlt 4,95 Euro und bekommt eine Festnetznummer des Ortes, für den man die Postadresse angibt. Die meisten Leute, die ich kennen haben eine Festnetz-Flat. Das bedeutet, sie können mich kostenlos auf dem Handy anrufen. Toll, oder?

Es ist ein Prepaid-Service. Das heißt, ich habe ein Guthaben, das ich vertelefonieren kann. Wenn das Guthaben weg ist, kann ich nicht mehr telefonieren. Das hat für mich vor allem den Vorteil, dass es keine versteckten Kosten gibt (bzw. diese micht nicht so hart treffen).

Für 10 Euro bekommt man eine 1-Monat-Festnetzflat.

Die Bestellung

Die Bestellung auf simquadrat.de ist extrem einfach.

Bestellung bei Simquadrat

Nach drei Schritten ist man wirklich fertig. Die beiden E-Mails, die man dann bekommt, sind auch schön kurz:

Simquadrat: Bestätigung
Simquadrat: Rechnung

Nach wenigen Tagen ist dann die Simkarte angekommen:

Brief, Seite 1
Brief, Seite 2
Simkarte von Sipgate

Wie man sieht, bekommt man eine Simkarte, wo man sich die richtige Größe herausbrechen kann. Micro- und Mini-Simkarten gehen also (bei Nano-Sim fürs iPhone bin ich mir nicht sicher, aber ich glaube auch das konnte man angeben).

Der Dienst

Simquadrat hat sofort funktioniert. Ich konnte telefonieren und SMS schreiben (die aber nicht bei meiner Freundin angekommen sind … aber das könnte auch gut an ihrem Handy/Anbieter liegen). Auch die Online-Benutzeroberfläche ist gut:

Simquadrat - So siehts aus, wenn man eingeloggt ist

ABER: Die Telefonqualität ist unterirdisch. Ich erwarte, dass der Zugangsanbieter mein Gespräch in so guter Qualität überträgt, wie es mein Handy unterstützt. Mit Simyo ist die Qualität genauso gut wie über das Festnetz-Telefon. Laut prepaid-wiki nutzt Simquadrat wie Simyo das E-Plus-Netz. Ich habe mit meiner Freundin mit meinem Nexus 4 am selben Tag (mit einem Abstand von 10 Minuten für dem Simkartenwechsel) einmal mit Simyo und einmal mit Simquadrat telefoniert. Das ist, im Bezug auf die Qualität der Sprachübertragung, ein himmelweiter Unterschied.

Fazit

Wenn man sehr günstig in Deutschland erreichbar sein will, ist Simquadrat super. Sonst ist Simyo besser.

DNS-Services

info@martin-thoma.de (Martin Thoma) — Sat, 20 Jul 2013 14:46:34 +0200

I’ve just read (ok, now it’s over 3 months ago) that Google Public DNS now supports DNSSEC (source).

I was curious what I currently use on my Linux Mint 14 machine. The relevant file is /etc/resolv.conf:

nameserver 127.0.1.1

# OpenDNS Fallback (configured by Linux Mint in /etc/resolvconf/resolv.conf.d/tail).
nameserver 208.67.222.222
nameserver 208.67.220.220

namebench

A programm called namebench checks how fast several DNS configurations would be for you.

namebench

SQL Injections

info@martin-thoma.de (Martin Thoma) — Thu, 11 Jul 2013 19:25:03 +0200

SQL is a language that allows prorammers to access data in databases. Most of the time (always?) you pass your queries in form of strings to the database. In online services it is quite common that the programmer formulates a template and the user fills in variables.

Example: IMDb

Take a look at IMDb. Users can search for movies by title:

IMDb: User Interface to search for a movie by title

When you search for “Harry Potter” for example, the following happens:

Search query within URL and results

You obviously interacted with imdb.com in a very dynamic way. The output of the website depends on what you typed in and IMDb has to search in its database for your search terms.

The programmers might have created a query that looks like this

SELECT * FROM `movie` WHERE title=$_GET['q']

Where $`_GET['q'] is your query.

Proof of concept

You need:

Apache Web Server
PHP
MySQL
PhpMyAdmin (for convenience)

When you search for “LAMP” (for Linux users) or for “WAMP” (for Windows users) you find a lot of information how to install this.

Place the following as hack.php in your web servers directory (might be /var/www):

<?
$mysqlhost = "localhost";
$mysqluser = "root";
$mysqlpwd = "asdfasdf";
$connection = mysql_connect($mysqlhost, $mysqluser, $mysqlpwd) or die
("Your connection string was wrong");
$db_selected = mysql_select_db('imdb', $connection);

$q = $_GET['q'];
if ($q != "") {
    $result = mysql_query("SELECT * FROM `movies` WHERE title='$q'");
    if (!$result) {
        die('MySQL query error: ' . mysql_error());
    }
    echo "Found ".mysql_num_rows($result)." movies:<br/>";
    while ($row = mysql_fetch_assoc($result)) {
        echo $row["id"].": ".$row["title"]."<br/>";
    }
}
?>

<form method="get" action="hack.php">
  <input type="text" name="q"/>
  <input type="submit" />
</form>

Now create a database called imdb with PHPMyAdmin and execute the following SQL:

CREATE TABLE IF NOT EXISTS `movies` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `title` varchar(255) NOT NULL,
  PRIMARY KEY (`id`)
) AUTO_INCREMENT=4;

INSERT INTO `movies` (`id`, `title`) VALUES
(1, 'Harry Potter'),
(2, 'Lord of the Rings'),
(3, 'Rise of the Silver Surfer');

CREATE TABLE IF NOT EXISTS `users` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `username` varchar(255) NOT NULL,
  `password` varchar(32) NOT NULL,
  `email` varchar(255) NOT NULL,
  PRIMARY KEY (`id`)
) AUTO_INCREMENT=3 ;

INSERT INTO `users` (`id`, `username`, `password`, `email`) VALUES
(1, 'admin', 'qewrtqwert', 'admin@imdb.com'),
(2, 'user', 'secret', 'mylittlepony@stupid.com');

Now go to http://localhost/hack.php. It should look like this:

Screenshot of my minimal example

When you search for “Harry Potter” it should show you “1: Harry Potter”. Note that there could be a lot of information, but I wanted to keep this example as small as possible.

Normal use of the web service

This resulted in the following query:

SELECT * FROM `movies` WHERE title='Harry Potter'

But a Hacker could also enter a string like this: ' OR '1'='1:

What a hacker could do

Even worse, the attacker could know that you use MySQL. Then he might know that MySQL uses INFORMATION_SCHEMA tables. He might enter this into the title input element:

' UNION SELECT table_name, table_type FROM information_schema.tables WHERE '1'='1

which results in this query:

SELECT * FROM `movies` WHERE title='' UNION SELECT table_name, table_type FROM information_schema.tables WHERE '1'='1'

which gives:

Attacker got all table names

This way, the attacker gets all table names from all databases on this machine. So he essentially can get everything stored in your database. And, of course, after getting everything he could drop it:

xkcd 327: Exploits of a mom

History

Just a few famous examples to show you that this happens all the time:

2005: USC admissions page (source)
2006: 800,000 datasets of personal information of students of UCLA (source)
2008, 2009: Heartland Payment Systems; 130 million credit and debit cards (source 1, source 2, see Albert Gonzalez)
2010: Royal Navy Website; Passwords and Usernames stolen (source)
2011: Sony:
LulzSec says it accessed the passwords, email addresses, home addresses and dates of birth of one million users. The group says it also stole all admin details of Sony Pictures, including passwords. 75,000 music codes and 3.5 million music coupons were also accessed, according to the press release.
(source)
2011: Expedia (source)
2011: MySql - Usernames and passwords stolen (source 1, source 2)
2011: Comodo (source 1, source 2)

Solutions

Sanitize user input, e.g. with mysql_real_escape_string
Use prepared statements
Switch of error reporting (this makes attacks more difficult, but doesn't prevent them)

Medion Life MD 83962 E69229

info@martin-thoma.de (Martin Thoma) — Tue, 09 Jul 2013 23:34:51 +0200

Momentan kann man diese Blootooth Lautsprecher für 30 Euro bei Aldi kaufen:

Medion MD 83962
Medion MD 83962: Anschlüsse
Medion MD 83962: Micro-USB Netzteil
Medion MD 83962: Micro-USB auf USB Anschluss

Ich habe sie mir angeschaut, bin aber enttäuscht. Wenn ich sie per Bluetooth an mein Nexus 4 anschließe gibt es hässliche Geräusche von sich. Außerdem ist die Audio-Qualität nicht so toll und die Freisprechfunktion ist … naja. Deutlich schechter als das interne Micro, wenn ich direkt zur Box rede.

Um euch mal einen Vergleich zu geben, habe ich die ersten zwei Minuten des Open Movie “Bick Buck Bunny” mit meinem Nexus 4 aufgenommen. Bei der ersten Aufnahme hört man die erste 2 Minuten durch meine internen Notebook-Lautsprecher, bei der zweiten durch den Medion Life MD 83962 Lautsprecher (per Kabel mit dem Notebook verunden):

Hier hört man keinen großen Unterschied. Aber die internen Lautsprecher meines Acer Travelmate 5744Z hören sich besser an als der Bluetooth-Lautsprecher. Und das Micro ist wie gesagt ziemlich schlecht. Ich kann also definitiv keine Kaufempfehlung geben.

Die Maße sind übrigens: 16,5 cm x 7,0 cm x 7,3 cm (Breite x Tiefe x Höhe).

Data Backup Strategies

info@martin-thoma.de (Martin Thoma) — Mon, 08 Jul 2013 08:48:50 +0200

Yesterday, I thought what would happen if my internal or external hard drive crashed.

The hard disk of this computer contains 53 GB of data (on Linux: df -H). As my home folder only contains 35.3 GB of data, 17.7 GB seem to be programs. 21.1 GB the remaining data is for movies and 6.1 GB are for programming (and backed up via GitHub). Hence the by far biggest part of the lost data would be movies, the rest would be scattered across my home folder.

The situation is similar for my external hard drive: A lot of video files, some audio files, A LOT of pictures and many, many miscellaneous files. I don’t want to lose data, but I don’t use my external HDD often. Most of the time I only have to do a RegEx search for file names (sometimes also a full text search for some files I found via RegEx) just to find out that the file I’m looking for is not there.

Online Services

I want to back up one computer with at least 200 GB which I don’t need to access often and 10 GB that need to access often.

A backup service should include software, that …

... allows me to configurate upload / dowload bandwidth limts
... should integrate into my os in such a way that it feels like using a hard disk
... lets me upload files of sizes up to 5 GB
... uploads in background automatically as soon as my notebook gets an internet connection
... uploads only parts of files, if only parts changed
... starts uploading as soon as a file changed
... creates checksums and compares them to check if files were correctly uploaded

A version control would be great, but that’s a feature I don’t expect. Also a possibility to share content (single files with a code) would be great, but I don’t know if any of those services offers that.

Name	Euro / Year	Clients	Information
Backblaze	37.06		Wiki
Carbonite	46.78		Wiki
CrashPlan	46.78		Wiki
IDrive	116.58		-
Jungle Disk	238.62	?	Wiki
MiMedia	77.20	?	Wiki
Mozy	131.00		Wiki
Nomadesk	93.57		-
SugarSync	194.94		Wiki
Ubuntu One	187.09		Wiki

I was a little bit surprised that there seems no way to use Google Drive as backup option (at least on Linux as there is this petition which was signed by 16,774 people).

What could go wrong?

A DOS attack against one of those companies might lead to temporarily unavailable services.
Financial problems, bad backup techniques or personal mistakes might lead to permanent loss of data.
Information might get leaked to the public.
Information might get leaked to government (PRISM, Tempora)
You might get analyzed by the company that stores your data. They could create a personal profile and sell that (maybe not even on the content you use, but perhaps only with filenames and edit times)
The software you've installed for your backup might be a malware.
The software you've installed might have security issues that allow attackers to execute malware.

Offline

RAID

All RAID levels (except for RAID 0) offer redundancy. This means, they store data on more than one hard disk. This way, you can restore data after on (or if maybe more) hard disk crashes. But this is by no mean a guarantee that your data is secure.

It’s recommended to use RAID with a RAID controller (a dedicated piece of hardware). Otherwise, you need some software that does it and your CPU time gets wasted with these operations.

What could go wrong:

One disk fails. You buy a new one to get security back, the RAID controller copies information to the new disk that replaced the crashed one. While it copies, the other disk gets heavy load. This heavy load might lead to another crash. Boom. Your data is lost.
Your computer might get damaged (e.g. by a fire, by an earthquake, by overvoltage, by a cup of tea you accidentally threw over it, by an act of agression)
A burglar would probably steal all hardware you have at home.

But RAID is no option for me as I use a notebook as my main computer.

External HDD

A very easy way to secure your files is an external hard disk drive. You can choose by yourself what and when to save your files for redundancy. But as I know myself, I will make those backups less often after a while. So I guess this will not work for me.

You could probably also make an external HDD raid.

More possibilities

Amazon S3 Glacier for 15.44 Euro/year or Amazon S3 Standard for 133.34 Euro/year e.g. with DragonDisk and others - transactions are not included!
Give external hard disks to friends / to bank.
Software:
- rsync for Linux
- Time machine for Mac

Improve German Public Transportation

info@martin-thoma.de (Martin Thoma) — Sat, 06 Jul 2013 20:45:32 +0200

Public transportation in Germany is much worse than it could be. I’ve got quite angry today because of that and thought about ways to improve the situation.

A story from today: I’ve bought two tickets to get from Karlsruhe to Augsburg (230 km) in about a month. I wanted to visit my dad in summer break just after an exam. I’ve bought a fixed connection to get there for “only” 35 Euro instead of 59 Euro. But I’ve made a mistake, my conenction started to early. Two hours after having bought the tickets, I wanted to exchange them for a connection that starts later. But I had to pay 15 Euro for exchanging them, 15 Euro for canceling the connection. And it got worse: I had to give them my home address to get at least some of the money back. For what the hell do they need my home address?

What could get improved?

Most people basically choose one company: Deutsche Bahn
It's too expensive.
Customer service of "Deutsche Bahn" is very bad.
Delays of at least 10 minutes always happen VERY often, delays of more than 30 minutes happen from time to time; I already had delay of more than 2 hours.

Alternative companies

Most people choose “Deutsche Bahn” (short: DB) because it is the biggest company that provides public transportation when you want to travel between cities. DB is well known and connects all big cities. However, there are alternatives.

After a law (§ 13 Abs. 2 of the Personenbeförderungsgesetz) was annulled, bus companies provided alternatives to traveling by train. Some companies are:

But they are difficult to find and people maybe don’t trust in them.

Informing people

Google Transit, FahrtenFuchs.de and Busliniensuche.de provide one possibility to inform people. But they all seem to have only a few bus companies.

Google Transit is interesting as they provide format specification for exchanging transit information.

What could be done

Contact bus companies and ask them if they want to provide transit information to the public in Google Transit Feed Format. You might want to hint them to Google Transit Partner Program.
- I've contacted "meinfernbus.de" (Saturday, 06.07.2013). Lets see if they answer.
- Contacted "KVV.de" (Saturday, 06.07.2013)
- Contacted "avv-augsburg.de" (Saturday, 06.07.2013)
Write software that allows bus companies to get their data into Google Transit Format
1. Write standard relational SQL database to store the required information
2. Create UI scribbles (e.g. with Balsamiq)
3. Write software

Some reactions

AVV (Augsburg)

My email from 06.07.2013 to kundencenter@avv-augsburg.de:

Sehr geehrte Damen und Herren, ich habe gerade gesehen, dass Google die Möglichkeit anbietet, die Fahrplaninformationen in Google Maps einzubinden: http://maps.google.com/help/maps/mapcontent/transit/index.html Könnten Sie das anbieten? Es ist vermutlich deutlich einfacher, die Fahrplanauskunft von Google zu nutzen als auf die AVV-Seite zu gehen. Mit freundlichen Grüßen, Martin Thoma

No reaction by now (21.07.2013)

KVV (Karlsruhe)

They have called me on 07.07.2013 and told me that they plan to bring the information to Google Maps, although they don’t know how long it will take.

MeinFernbus.de

My email from 06.07.2013:

Sehr geehrte Damen und Herren, ich habe mich heute sehr über die Deutsche Bahn geärgert und auf der Suche nach Alternativen meinfernbus.de gefunden. Allerdings habe ich zuerst über Google Maps nach Busverbindungen zwischen Karlsruhe und Augsburg gesucht, wo mir leider nur die Bahn angeboten wurde. Haben Sie schon von Google Transit<http://maps.google.com/help/maps/mapcontent/transit/index.html>gehört? Was halten Sie davon, an diesem Programm teilzunehmen? Mit freundlichen Grüßen, Martin Thoma

Answer from 08.07.2013:

Sehr geehrter Herr Thoma, bitte entschuldigen Sie die späte Antwort. Vielen Dank für Ihre Email, Ihre Kooperationsanfrage ist bei uns eingegangen. Wir freuen uns sehr, dass Sie mit MeinFernbus.de, dem Marktführer im deutschen Fernbusverkehr, zusammenarbeiten möchten. Wir prüfen gerne eine entsprechende Zusammenarbeit. Zu gegebener Zeit setzt sich unser Kooperationsmanager mit Ihnen in Verbindung. Aufgrund der Vielzahl eingehender Anfragen bitten wir Sie zunächst noch um etwas Geduld. Vielen Dank für Ihr Verständnis, herzliche Grüße aus der Hauptstadt Friederike Freytag

Graphic filters

info@martin-thoma.de (Martin Thoma) — Thu, 04 Jul 2013 18:21:19 +0200

I begin to fall in love with JavaScript and HTML5. You can access your Webcam with JS! As an example, I’ve implemented some graphic filters.

Basics

HTML5

You need:

This is the bare minimum HTML code you need for valid HTML: ```html <!DOCTYPE html> Some title ```
JavaScript
Important functions / datastructures are:
- getUserMedia: see support by browsers
- ImageData: WTF? The image is a ONE dimensional array of integers in 0, ..., 255. So the first 4 array elements describe the pixel (0|0) with its RGBA value
A starting point for your code might be: ```javascript 'use strict'; var video = document.querySelector("#vid"); var canvas = document.querySelector('#canvas'); var context = canvas.getContext('2d'); var localMediaStream = null; var onCameraFail = function (e) { console.log('Camera did not work.', e); }; setInterval(function snapshot() { if (localMediaStream) { context.drawImage(video, 0, 0); var width = 640; var height = 480; var imgDataNormal = context.getImageData(0, 0, width, height); var imgData = context.createImageData(width, height); // convert image to grayscale for (var i = 0; i < imgData.width*imgData.height*4; i += 4) { var r = imgDataNormal.data[i + 0]; var g = imgDataNormal.data[i + 1]; var b = imgDataNormal.data[i + 2]; var brightness = (3*r+4*g+b)>>>3; imgData.data[i] = brightness; imgData.data[i+1] = brightness; imgData.data[i+2] = brightness; } context.putImageData(imgData, 0, 0); } }, 500); navigator.getUserMedia = navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia || navigator.msGetUserMedia; window.URL = window.URL || window.webkitURL; navigator.getUserMedia({video:true}, function (stream) { video.src = window.URL.createObjectURL(stream); localMediaStream = stream; }, onCameraFail); console.log(localMediaStream); ```
Interactive example

You need a webcam for this: Open demonstration in new window
This is what it should look like:
Webcam example
And it gives these results:
- Prewitt x-filter example
- Prewitt y-filter example
- Laplace filter example
By the way, you can check if a website is currently accessing your webcam (with Google Chrome):
Webcam indicator on tab in Google Chrome
If you want to use these examples from your Android phone, you might have to enable getUserMedia. To do this, enable "Web RTC" in "chrome://flags":
Enamble Web-RTC in Chrome for Android

three.js is AWESOME!

info@martin-thoma.de (Martin Thoma) — Mon, 01 Jul 2013 16:21:44 +0200

I recently discovered three.js, a JavaScript library/API used to create and display animated 3D computer graphics.

Here are some examples what you can do with three.js. It’s pure JavaScript, no Flash required!

They all worked smooth on my Notebook (Intel Pentium P6200 processor) with Google Chrome 28 on Linux.

You have to click on the links to see the examples!

Stemkoski

Textures

Three.js textures example

Demonstration

Three.js

Threejs Skybox example

Demonstration

Reflection

Reflection example

Demonstration

Webcam

Webcam example

Demonstration

Motion detection

Motion detection with three.js!

Demonstration

3D function plotter

Demonstration

HexGL

Demonstration

Solar System

Solar system simulation

Demonstration

Planet Maker

PlanetMaker

Demonstration - takes ages to load, but when its loaded it is fast

Music videos

info@martin-thoma.de (Martin Thoma) — Sun, 30 Jun 2013 21:21:51 +0200

Paint: Lord of the Rings

Chatroulette Concert

Chatroulette Love Song

28 Cartoon Theme Songs in 7 Minutes

Bodo Wartke - Da muss er durch

Liebeslied (In German, English, French, Spanish, Italian, Chinese, Russian, Turkish, Arabic, Bavarian, Swiss, Finnish)
Believe in Steve

Atomic Theory Song

The 50 States Of The USA and The Capitals

(From television series “Animaniacs”)

Spline interpolation

info@martin-thoma.de (Martin Thoma) — Sun, 30 Jun 2013 13:58:20 +0200

Just like before with polynomial interpolation, we have a list of $n+1$ given point $(x_i, y_i)$ with $x_0 < x_1 < \dots < x_n$ .

We want to find a function that goes through those points and approximates the underlying function that produced that points as good as possible.

Polynomial interpolation

The problem of polynomial interpolation were oscillations at the end of the interval you wanted to interpolate (see interactive example):

Oscillations you get with polynomial interpolation

As you can see, polynomial interpolation with equally spaced points is very, very bad at the ends of the interval. Tschebyscheff spaced points are much better, but you can still see that the interpolated function is different from the original.

Splines

A way to solve this problem are splines. A spline is a piecewise-defined function that goes through some points (aka knots) and is smooth.

More formally: Let $s: [x_0,x_n] \rightarrow \mathbb{R}$ be a spline. Then:

(S1) cubic: `$\forall i \in \{1, \dots, n\}: s|_{x_{i-1}, x_i}$` is a cubic function
(S2) interpolation: `$\forall i \in \{0, \dots, n\}: s(x_i) = y_i$`
(S3) smooth: `$s \in C^2([x_0, x_n])$` and `$\int_{x_0}^{x_n} s''(x)^2 \mathrm{d}x$` is minimal

When you use a cubic function $a x^3 + b x^2 + cx + d$ for each of the $n$ intervals that we got by our $n+1$ points, you have $4n$ variables that you need to calculate.

Condition (S2) gives two equations per interval which makes $2n$ equations of the form:

$ \begin{align} y_i &= a_i x_i^3 &&+ b_i x_i^2 &&+ c_i x_i &&+ d_i\\ y_{i+1} &= a_i x_{i+1}^3 &&+ b_i x_{i+1}^2 &&+ c_i x_{i+1} &&+ d_i \end{align} $

At first glance condition (S3) - $s \in C^2([x_0, x_n])$ - seems to be redundant with (S1) - $s$ is piecewise cubic. Every polynomial is in $C^\infty(\mathbb{R})$ , so it certainly is in $C^2([x_0, x_n])$ .
That’s correct. But $s$ is not a polynomial. It’s only piecewise-defined as a polynomial. That makes a difference at the ends of the intervals. And it gives us $2n-2$ more equations:

$\displaystyle \begin{align} s_i' (x_{i}) &= s_{i+1}'(x_{i}) \;\;\; &&\forall i=1, \dots, n-1\\ s_i''(x_{i}) &= s_{i+1}''(x_{i}) \;\;\; &&\forall i=1, \dots, n-1 \end{align} $

which is equivalent to

$\displaystyle \begin{align} 3a_i x_i^2 + 2b_i x_i + c_i &= 3a_{i+1} x_i^2 + 2b_{i+1} x_i + c_{i+1} \;\;\; &&\forall i=1, \dots, n-1\\ 6a_i x_i + 2b_i &= 6a_{i+1} x_i + 2b_{i+1} \;\;\; &&\forall i=1, \dots, n-1 \end{align} $

All equations we have are linear. Please note that the variables we want to determine are $a_i, b_i, c_i, d_i$ . So $x_i^3$ is simply a multiplicative constant that we have to evaluate before we solve our system of equations.

But at the moment, we only have $2n+2\cdot(n-1) = 4n -2$ equations, but we have $4n$ variables. So we need ancillary conditions to solve this linear system of equations.

Possible ancillary conditions

natural splines: `$s''(x_0) =0, \;\;\; s''(x_n) = 0$`
clamped splines: `$s'(x_0) = f'(x_0),\;\;\; s'(x_n)= f'(x_n)$` where `$y_0'$` and `$y_n'$` can be any value
periodic: `$s'(x_0) = s'(x_n), \;\;\; s''(x_0) = s''(x_n)$`
not-a-knot: `$s_1''' = s_2''', \;\;\; s_{n-1}''' = s_{n}'''$`

George MacKerron shows how the results can differ in his article Cubic splines in JavaScript (via CoffeeScript):

Different results for different ancillary conditions

Code for natural splines

I will store splines as a list of maps. Each map is one piece of the spline and has:

`$u$`: Start of the interval
`$v$`: End of the interval
`$a,b,c,d$`: cubic function `$ax^3 + bx^2 + cx +d$`

Please note that I didn’t test the code below. It’s likely that there are errors with indices.

#!/usr/bin/env python
# -*- coding: utf-8 -*-

def niceCubicPolynomial(p):
    tmp = ""
    if p["a"] == 1:
        tmp += " x^3"
    elif p["a"] != 0:
        tmp += "%.2fx^3" % p["a"]
    if p["b"] == 1:
        tmp += "\t+ x^2"
    elif p["b"] != 0:
        tmp += "\t+ %.2fx^2" % p["b"]
    else:
        tmp += "\t\t"
    if p["c"] == 1:
        tmp += "\t+ x"
    elif p["c"] != 0:
        tmp += "\t+ %.2fx" % p["c"]
    else:
        tmp += "\t\t"
    if p["d"] != 0:
        tmp += "\t+ %.2f" % p["d"]
    return tmp

def getSpline(points):
    """ points should be a list of maps, 
        where each map represents a point and has "x" and "y" """
    import numpy, scipy.linalg

    # sort points by x value
    points = sorted(points, key=lambda point: point["x"])

    n = len(points) - 1

    # Set up a system of equations of form Ax=b
    A = numpy.zeros(shape=(4*n,4*n))
    b = numpy.zeros(shape=(4*n,1))

    for i in range(0, n):
        # 2n equations from condtions (S2)
        A[i][4*i+0] = points[i]["x"]**3
        A[i][4*i+1] = points[i]["x"]**2
        A[i][4*i+2] = points[i]["x"]
        A[i][4*i+3] = 1
        b[i] = points[i]["y"]

        A[n+i][4*i+0] = points[i+1]["x"]**3
        A[n+i][4*i+1] = points[i+1]["x"]**2
        A[n+i][4*i+2] = points[i+1]["x"]
        A[n+i][4*i+3] = 1
        b[n+i] = points[i+1]["y"]

        # 2n-2 equations for (S3):
        if i == 0:
            continue
        # point i is an inner point
        A[2*n+(i-1)][4*(i-1)+0] = 3*points[i]["x"]**2
        A[2*n+(i-1)][4*(i-1)+1] = 2*points[i]["x"]
        A[2*n+(i-1)][4*(i-1)+2] = 1
        A[2*n+(i-1)][4*(i-1)+0+4] = -3*points[i]["x"]**2
        A[2*n+(i-1)][4*(i-1)+1+4] = -2*points[i]["x"]
        A[2*n+(i-1)][4*(i-1)+2+4] = -1
        b[2*n+(i-1)] = 0

        A[3*n+(i-1)][4*(i-1)+0] = 6*points[i]["x"]
        A[3*n+(i-1)][4*(i-1)+1] = 2
        A[3*n+(i-1)][4*(i-1)+0+4] = -6*points[i]["x"]
        A[3*n+(i-1)][4*(i-1)+1+4] = -2
        b[3*n+(i-1)] = 0
    # Natural spline:
    A[3*n-1+0][0+0] += 6*points[0]["x"]
    A[3*n-1+0][0+1] += 2
    b[3*n-1+0] += 0

    A[3*n+n-1][4*(n-1)+0] += 6*points[n]["x"]
    A[3*n+n-1][4*(n-1)+1] += 2
    b[3*n+n-1] += 0

    x = scipy.linalg.solve(A, b)
    spline = []
    for i in range(0, n):
        spline.append({"u": points[i]["x"], "v": points[i+1]["x"],
                        "a": float(x[4*i+0]),
                        "b": float(x[4*i+1]),
                        "c": float(x[4*i+2]),
                        "d": float(x[4*i+3])})
    return spline

if __name__ == "__main__":
    points = []
    points.append({"x": 0.0, "y": -4})
    points.append({"x": 1.0, "y": 9})
    points.append({"x": 2.0, "y": 35})
    points.append({"x": 3.0, "y": 70})
    spline = getSpline(points)
    for p in spline:
        tmp = "[%.2f, %.2f]:" % (p["u"], p["v"])
        tmp += niceCubicPolynomial(p)
        print(tmp)

Zero Mean Normalized Cross-Correlation

info@martin-thoma.de (Martin Thoma) — Fri, 28 Jun 2013 22:15:45 +0200

An image from Tsukuba University. This is one of hundreds of images that you can use to test your algorithms. Link is below.

Zero Mean Normalized Cross-Correlation or shorter ZNCC is an integer you can get when you compare two grayscale images.

Lets say you have a webcam at a fixed position for security. It takes images all the time, but most of the time the room is empty. So quite a lot of images will not be interesting. They only waste space. So you want to get rid of those redundant images.

BUT those images are not identical! Even if the scenery didn’t change, your sensor will produce slightly different results. A human will not notice them, but you can’t simply compare images bit by bit. Even if you could, the images will be different because the sun moved (and so do shadows) and perhaps you have a clock in the image.

Now you can solve this problem with various techniques.

I want to describe those techniques in a very general way. As the images in other scenarios might have different sizes and you probably don’t want to compare whole images, I’ll assume you have a part of both image of size $(2n+1) \times (2n+1)$ . The pixel in the center has coordinates $(u_1, v_1)$ for the part of the first image and $(u_2, v_2)$ for the second image.

Sum of squared differences

Go through all pixels, get the difference of both and add up the squares:

$\displaystyle SSD(Img_1, Img_2, u_1, v_1, u_2, v_2, n) := \sum_{i=-n}^n \sum_{j=-n}^n \left ( Img_1(u_1+i, v_1+j) - Img_2(u_2 + i, v_2 + j) \right )^2$

When SSD is small, both images are very similar. Wehn SSD is 0, the images are identical.

Zero Mean Normalized Cross-Correlation

The average gray value is: $\displaystyle \overline{Img}(u, v, n) := \frac{1}{(2n+1)^2} \sum_{i=-n}^n \sum_{j=-n}^n Img(u+i, v+j)$

The standard deviation is: $\displaystyle \sigma(u, v, n) := \sqrt{\frac{1}{(2n+1)^2} \left (\sum_{i=-n} \sum_{j=-n}^n (Img(u +i, v+j)-\overline{Img}(u, v, n))^2 \right )}$

The ZNCC is defined as:

$\displaystyle ZNCC(Img_1, Img_2, u_1, v_1, u_2, v_2, n) := \frac{\frac{1}{(2n+1)^2}\sum_{i=-n}^n \sum_{j=-n}^n \prod_{t=1}^2 \left (Img_t (u_t+i,v_t+j) - \overline{Img}(u_t, v_t, n) \right )}{\sigma_1(u_1, v_1, n) \cdot \sigma_2(u_2, v_2, n)}$

The higher the ZNCC gets, the more are those two images correlated. (I think the value is always in [0, 1])

Here is some Python code:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

def getAverage(img, u, v, n):
    """img as a square matrix of numbers"""
    s = 0
    for i in range(-n, n+1):
        for j in range(-n, n+1):
            s += img[u+i][v+j]
    return float(s)/(2*n+1)**2

def getStandardDeviation(img, u, v, n):
    s = 0
    avg = getAverage(img, u, v, n)
    for i in range(-n, n+1):
        for j in range(-n, n+1):
            s += (img[u+i][v+j] - avg)**2
    return (s**0.5)/(2*n+1)

def zncc(img1, img2, u1, v1, u2, v2, n):
    stdDeviation1 = getStandardDeviation(img1, u1, v1, n)
    stdDeviation2 = getStandardDeviation(img2, u2, v2, n)
    avg1 = getAverage(img1, u1, v1, n)
    avg2 = getAverage(img2, u2, v2, n)

    s = 0
    for i in range(-n, n+1):
        for j in range(-n, n+1):
            s += (img1[u1+i][v1+j] - avg1)*(img2[u2+i][v2+j] - avg2)
    return float(s)/((2*n+1)**2 * stdDeviation1 * stdDeviation2)

if __name__ == "__main__":
    A  = [[1,2,3],[4,5,6],[7,8,9]]
    B1 = [[1,2,3],[4,5,6],[7,8,9]]
    B2 = [[1,2,3],[4,5,6],[7,8,7]]
    print(zncc(A, B1, 1,1,1,1, 1))
    print(zncc(A, B2, 1,1,1,1, 1))

JavaScript: WTF?!?

info@martin-thoma.de (Martin Thoma) — Wed, 26 Jun 2013 22:51:52 +0200

JavaScript is THE web programming language. It gets interpreted by your browser and web applications like Google Mail, Google Maps and Facebook make heavy use of them. Almost always, when you see something working smoothly / interactive, you see JavaScript in action.

But JavaScript has a lot of “features” which are … well, I don’t have words for those. Just continue reading.

Most of the following content is from a StackOverflow question Strangest language feature

Weak typing

Example

console.log(3..toString());

Output

'3'

Explanation 3. is a floating point and can get converted to a string. But 3.toString() gives

SyntaxError: Unexpected token ILLEGAL

Weak typing and string concatenation

Example

console.log('5' + 3);
console.log('5' - 3);

Output

'53'
2

Explanation JavaScript automatically converts datatypes and + is used for string concatenation and for addition.

In the first case, as the first datatype is a string and + is defined for strings as concatenation, JS converts 3 to '3' which results in the string '53'.

In the second case, - is only defined for subtraction so '5' gets converted to a number (int? float? I don’t know. I guess int.)

Automatic semicolons

Example 1

function test() {
    return
    {
        id : 1338,
        title : 'This is a test'
    };
}

console.log(test());

Example 2

function test() {
    return
        2 + 2;
}

console.log(test());

Output 1

Uncaught SyntaxError: Unexpected token :

Output 2

undefined

Explanation JS adds a ; at every line end. Automatically. You can’t prevent it.

Please note that the second output is no error! It has a (valid) return value of undefined.

Truth table

Example

console.log("------- 0 == XYZ ---------");
console.log(0 == 0);
console.log(0 == false);
console.log(0 == '');
console.log(0 == null);
console.log(0 == undefined);
console.log(0 == " \t\r\n");
console.log(0 == '0');
console.log(0 == 'false');
console.log("------- false == XYZ ---------");
console.log(false == false);
console.log(false == '');
console.log(false == null);
console.log(false == undefined);
console.log(false == " \t\r\n");
console.log(false == '0');
console.log(false == 'false');
console.log("------- '' == XYZ ---------");
console.log('' == '');
console.log('' == null);
console.log('' == undefined);
console.log('' == " \t\r\n");
console.log('' == '0');
console.log('' == 'false');
console.log("------- null == XYZ ---------");
console.log(null == null);
console.log(null == undefined);
console.log(null == " \t\r\n");
console.log(null == '0');
console.log(null == 'false');
console.log("------- undefined == XYZ ---------");
console.log(undefined == undefined);
console.log(undefined == " \t\r\n");
console.log(undefined == '0');
console.log(undefined == 'false');
console.log("-------  \t\r\n == XYZ ---------");
console.log(" \t\r\n" == " \t\r\n");
console.log(" \t\r\n" == '0');
console.log(" \t\r\n" == 'false');
console.log("------- '0' == XYZ ---------");
console.log('0' == '0');
console.log('0' == 'false');
console.log("------- 'false' == XYZ ---------");
console.log('false' == 'false');
console.log("--------------------");
console.log("-------  XYZ == 0 ---------");
console.log(0 == 0);
console.log(false == 0);
console.log('' == 0);
console.log(null == 0);
console.log(undefined == 0);
console.log(" \t\r\n" == 0);
console.log('0' == 0);
console.log('false' == 0);
console.log("-------  XYZ == false ---------");
console.log(false == false);
console.log('' == false);
console.log(null == false);
console.log(undefined == false);
console.log(" \t\r\n" == false);
console.log('0' == false);
console.log('false' == false);
console.log("-------  XYZ == '' ---------");
console.log('' == '');
console.log(null == '');
console.log(undefined == '');
console.log(" \t\r\n" == '');
console.log('0' == '');
console.log('false' == '');
console.log("-------  XYZ == null ---------");
console.log(null == null);
console.log(undefined == null);
console.log(" \t\r\n" == null);
console.log('0' == null);
console.log('false' == null);
console.log("-------  XYZ == undefined ---------");
console.log(undefined == undefined);
console.log(" \t\r\n" == undefined);
console.log('0' == undefined);
console.log('false' == undefined);
console.log("-------  XYZ == \t\r\n ---------");
console.log(" \t\r\n" == " \t\r\n");
console.log('0' == " \t\r\n");
console.log('false' == " \t\r\n");
console.log("-------  XYZ == '0' ---------");
console.log('0' == '0');
console.log('false' == '0');
console.log("-------  XYZ == 'false' ---------");
console.log('false' == 'false');

Output

==	0	false	''	null	undefined	" \t\r\n"	'0'	'false'
0	true	true	true	false	false	true	true	false
false	true	true	true	false	false	true	true	false
''	true	true	true	false	false	false	false	false
null	false	false	false	true	true	false	false	false
undefined	false	false	false	true	true	false	false	false
" \t\r\n"	true	true	false	false	false	true	false	false
'0'	true	true	false	false	false	false	true	false
'false'	false	false	false	false	false	false	false	true

Explanation Wow. This is fucked up. I’ve expected that strings compared to anything that is not the string itself evaluates to false.

At least the table is symmetric and the diagonal is true. Please also note that JavaScript has a === operator.

The truth

Example

console.log(2 == [2]);
console.log(2 == [[2]]);

Output

true
true

Explanation At StackOverflow

Date

Example

var futureDate = new Date(2010,77,154);
console.log(futureDate);
console.log(futureDate.getYear());

Output

Tue Nov 01 2016 00:00:00 GMT+0100 (CET) 
116

Explanation 77 months and 154 days from the 0th day of 0th month of 2010

You should use getFullYear() instead of getYear(), as the later one returns year - 1900 (Why? When is this useful?)

Integer overflow

Example

console.log(111111111111111111111);

Output

111111111111111110000

Explanation It’s ok that JavaScript fails at handling this integer. I think it converts this to a float, but I’m not sure about that. But no matter what it does here, it doesn’t throw an error. That’s bad. How is a developer supposed to know in a big application know when something went wrong?

Function parameters

In Javascript, declaring the parameters a function accepts is only a convenience to the programmer. All variables passed through the function call are accessible by the keyword arguments. So the following would alert "world":
function blah(){
    alert(arguments[1]);
}

blah("hello", "world");

Source on StackOverflow

Global variables

Example

var a = 12;

function test() {
  a = 1337;
}
test();
console.log(a);

Output

Explanation When you forget to use var inside of test() you might accidentally use a global variable.

Polynomial interpolation

info@martin-thoma.de (Martin Thoma) — Sat, 22 Jun 2013 18:41:43 +0200

Suppose you have a list of $n+1$ given point $(x_i, y_i)$ with $i \in \{0, \dots, n\}$ and $\forall i,j \in \{0, \dots, n\}: i \neq j \Rightarrow x_i \neq x_j$ .

Now you want to find a polynomial $\displaystyle p(x) = \sum_{i=0}^n a_i \cdot x^i$ that goes through all of the given points. This means:

$\forall i \in \{0, \dots, n\}: p(x_i) = y_i$

Existence and uniqueness

Theorem: When you have a list of $n+1$ points with mutually different $x_i$ there is exactly one polynom of degree $\leq n$ that fits those points.

Proof:

You can formulate a linear system of equations:

$\underbrace{\begin{pmatrix} x_0^0 & \cdots & x_0^n \\ \vdots & \ddots & \vdots \\ x_n^0 & \cdots & x_n^n \end{pmatrix} }_{:= A \in \mathbb{R}^{(n+1) \times (n+1)}} \begin{pmatrix} a_0 \\ \vdots \\ a_n \end{pmatrix} = \begin{pmatrix} y_0 \\ \vdots \\ y_n \end{pmatrix}$

A matrix of the form of $A$ is called Vandermonde matrix. When the data points $x_i$ are mutually different, it is known that the Vandermonde matrix is invertible (source).

This means, the solution $(a_0 \dots a_n)^T$ of this linear equation gives the polynom $p(x) = \sum_{i=0}^n a_i x^i$

So the solution exists and is unique $\blacksquare$

Straight forward interpolating polynomials

For this algorithm, I’ll find the polynomial in its monomial from $p(x) = \sum_{i=0}^n a_i x^i$ . I’ll use the matrix $A$ from section “Uniqueness”.

You might want to take a look at my article about Gaussian elimination.

#!/usr/bin/env python
# -*- coding: utf-8 -*-
  
def pprintGaus(A):
    """ Pretty print a n&times;n matrix with a result vector n&times;1. """
    n = len(A)
    for i in range(0, n):
        line = ""
        for j in range(0, n+1):
            line += str(A[i][j]) + "\t"
            if j == n-1:
                line +=  "| "
        print(line)
    print("")

def pprintPolynomial(A):
    """ Pretty print a polynomial. """
    line = ""
    for i in range(len(x)-1, -1, -1):
        if x[i] != 0:
            if i == 0:
                line += "+" + str(x[i])
            else:
                if x[i] == 1:
                    line += "+ x^" + str(i) + "\t"
                elif x[i] == -1:
                    line += "- x^" + str(i) + "\t"
                else:
                    line += "+" + str(x[i]) + "&middot;x^" + str(i) + "\t"
    print(line)

def gauss(A):
    """ Solve a linear sysem of equations given by a n&times;n matrix 
        with a result vector n&times;1. """
    n = len(A)
  
    for i in range(0,n):
        # Search for maximum in this column
        maxEl = abs(A[i][i])
        maxRow = i
        for k in range(i+1,n):
            if abs(A[k][i]) > maxEl:
                maxEl = A[k][i]
                maxRow = k
  
        # Swap maximum row with current row (column by column)
        for k in range(i,n+1):
            tmp = A[maxRow][k]
            A[maxRow][k] = A[i][k]
            A[i][k] = tmp
  
        # Make all rows below this one 0 in current column
        for k in range(i+1,n):
            c = -A[k][i]/A[i][i]
            for j in range(i,n+1):
                if i==j:
                    A[k][j] = 0
                else:
                    A[k][j] += c * A[i][j]
  
    # Solve equation Ax=b for an upper triangular matrix A
    x=[0 for i in range(n)]
    for i in range(n-1,-1,-1):
        x[i] = A[i][n]/A[i][i]
        for k in range(i-1,-1,-1):
            A[k][n] -= A[k][i] * x[i]
    return x

def setGauss(points):
    """ Create a system of equations for gaussian elimination from 
        a set of points. """
    n = len(points) - 1
    A = [[0 for i in range(n+2)] for j in range(n+1)]
    for i in range(n+1):
        x = points[i]["x"]
        for j in range(n+1):
            A[i][j] = x**j
        A[i][n+1] = points[i]["y"]
    return A
  
if __name__ == "__main__":
    from fractions import Fraction
  
    # Read input data
    points = []
    points.append({"x": Fraction(-1), "y": Fraction(1)})
    points.append({"x": Fraction(1), "y": Fraction(1)})
    points.append({"x": Fraction(2), "y": Fraction(2)})

    A = setGauss(points)
  
    # Print input
    pprintGaus(A)
  
    # Calculate solution
    x = gauss(A)
  
    # Print result
    pprintPolynomial(x)

It is also interesting to get the value of $p(x)$ at any given point $x \in \mathbb{R}$ :

def evaluatePolynomial(p, x):
    y = 0
    xi = 1
    for i, a in enumerate(p):
        y += a * xi
        xi *= x
    return y

Time complexity to get the polynomial: $\frac{1}{3} n^3 + \mathcal{O}(n^2)$ (where $n$ is the number of points) Space complexity for the polynomial: $\mathcal{O}(n^2)$ (where $n$ is the number of points)

Time complexity to evaluate the value of $p(x)$ for any $x \in \mathbb{R}$ : $\mathcal{O}(n)$ .

Polynomials

$\displaystyle \mathbb{R}_n[X] := \left \{p:\mathbb{R} \rightarrow \mathbb{R} | p(x) = \sum_{i=0}^n a_i \cdot x^i \text{ with } a_0, \dots, a_n \in \mathbb{R} \right \}$

So $\mathbb{R}_n[X]$ are all polynomials with real coefficients and degree not higher than latex]n $. $ \mathbb{R}_n[X] $ forms a vector space for $ n \in \mathbb{N}_0 $. The degree of that vector space is $ n+1`$.

What does that mean?

This means you can use different bases for polynomials. The base we usually use for $\mathbb{R}_n[X]$ is $\{x^0, x^1, x^2, x^3, \dots\}$, but you can switch the base.

Lagrange interpolation

Define $n+1$ polynomials like this:

$\displaystyle L_{i}(x) := \prod_{j=0 \atop j \neq i}^n \frac{x-x_j}{x_i - x_j}$

This means:

$\displaystyle L_{i}(x_i) := \prod_{j=0 \atop j \neq i}^n \frac{x_i-x_j}{x_i - x_j} = 1$

and

$\displaystyle L_{i}(x_p) := \prod_{j=0 \atop j \neq i}^n \frac{x_p-x_j}{x_i - x_j} = 0 \;\; p \in \{0, \dots, n\} \setminus \{i\}$

So the polynomial $y_i \cdot L_{i}(x)$ fits the point $(x_i, y_i)$ and is zero for all other points. This implies that

$\displaystyle p(x) = \sum_{i=0}^n y_i \cdot L_i(x)$ is an interpolation of our data points. The degree of $p(x)$ is not higher than $n$. You can see that easily when you take a look at the definition of $p(x)$ and $L_i(x)$.

The polynomials $L_i(x)$ form another base for $\mathbb{R}_n[X]$.

Lagranges way to interpolate polynomials can be implemented like this:

def lagrangeInterpolation(points):
    p = []
    for i in range(len(points)):
        Li = {"y": points[i]["y"], "polynomial":[]}
        for j in range(len(points)):
            if j == i:
                continue
            Li["polynomial"].append({
                "sub": points[j]["x"], 
                "divisor": points[i]["x"] - points[j]["x"]
            })
        p.append(Li)
    return p

def evaluateLagrangePolynomial(p, x):
    y = 0
    for Li in p:
        prod = 1
        for term in Li["polynomial"]:
            prod *= (x - term["sub"])/term["divisor"]
        y += Li["y"]*prod
    return y

Time complexity to get the polynomial: $n^2 + \mathcal{O}(n)$ (where $n$ is the number of points) Space complexity for the polynomial: $\mathcal{O}(n^2)$ (where $n$ is the number of points)

Time complexity to evaluate the value of $p(x)$ for any $x \in \mathbb{R}$: $\mathcal{O}(n^2)$.

Newton interpolation

Define

$\displaystyle N_i(x) := \prod_{j=0}^{i-1} (x-x_j)$

So you know that

$N_i(x) = 0 \Leftrightarrow \exists p \in \{1, \dots, i\}: x = x_{i-p}$

So your polynomial is

$\displaystyle p(x) = \sum_{i=0}^n c_i \cdot N_i(x)$

for the correct $c_i$. You can do this by solving the following system of equations. Please note that you don’t have to store that matrix to get those $c_i$, divided differences are better.

$\begin{pmatrix} 1 & & & & 0 \\ 1 & (x_1 - x_0) & & & \\ 1 & (x_2 - x_0) & (x_2 - x_0)(x_2 - x_1) & & \\ \vdots & \vdots & & \ddots & \\ 1 & (x_n - x_0) & \cdots & & \prod_{i=0}^{n-1}(x_n - x_i) \\ \end{pmatrix} \cdot \begin{pmatrix} c_0 \\ \vdots \\ c_n \end{pmatrix} = \begin{pmatrix} f_0 \\ \vdots \\ f_n \end{pmatrix}$

You can get this lower triangular matrix like this:

def getGaussSystemForNewton(points):
    n = len(points) - 1
    A = [[0 for i in range(n+2)] for j in range(n+1)]
    for j in range(0,n+2):
        for i in range(j,n+1):
            if j == 0:
                A[i][j] = 1
            else:
                A[i][j] = A[i][j-1]*(points[i]["x"]-points[j-1]["x"])
        if j == n+1:
            for i in range(0, n):
                A[i][j] = points[i]["y"]
    return A

From my previous posts about solving equations of upper triangular matrices and lower unitriangular matrices you know that the space complexity of this is in $\Theta(n^2)$.

According to Wikipedia, you can use Horner’s method to evaluate this Polynom in $`\mathcal{O}(n)$. But I really don’t want to implement this today.

Interactive example

Click to add points.
Ctrl+Click to remove point.
You can enter a function that is valid JavaScript in the text field.

Mathematische Strukturen

info@martin-thoma.de (Martin Thoma) — Sun, 16 Jun 2013 18:14:57 +0200

Es gibt einen ganzen Haufen an mathematischen Strukturen. Dieser Artikel soll jeweils die Definition und bekannte Beispiele sammeln. Weitere Strukturen bzw. Beispiele können gerne in den Kommentaren genannt werden.

Magma

Sei `$M$` eine Menge und `$*:M \times M \rightarrow M$` eine auf `$M$` abgeschlossene Abbildung. Dann heißt `$(M,*)$` ein Magma.

Sei `$(M,*)$` ein Magma. `$(M,*)$` heißt kommutativ `$:\Leftrightarrow \forall a, b \in M: a*b = b*a$`.

Ein Synonym zu „kommutativ“ ist „abelsch“.

Beispiele:

Jede Halbgruppe ist auch ein Magma.
`$(\mathbb{Z}, -)$` ist ein Magma, aber keine Halbgruppe: `$1-(1-1) = 1-0 = 1 \neq -1 = 0-1 = (1-1)-1$`
`$(\mathbb{R} \setminus \{0\}, : )$` ist ein Magma, aber keine Halbgruppe: `$1 : (7:7)=1:1=1 \neq \frac{1}{49} = \frac{1}{7} : 7 = (1:7):7$`

Gegenbeispiele:

`$(\mathbb{N}, : )$` ist nicht abgeschlossen, also kein Magma.
`$(\mathbb{N}, - )$` ist nicht abgeschlossen, also kein Magma.

Halbgruppe

Sei `$(M, *)$` ein Magma. `$(M, *)$` heißt Halbgruppe `$:\Leftrightarrow (M, *)$` ist assoziativ.

Eine Verknüpfung $*$ ist genau dann assoziativ auf einer Menge $M$ , wenn gilt: $\forall a,b,c \in M: (a*b)*c = a*(b*c)$

Beispiele:

Jedes Monoid ist auch eine Halbgruppe.
`$(\mathbb{N}_{\geq 1}, +)$` ist eine Halbgruppe, aber kein Monoid.

Monoid

Sei `$(M, *)$` eine Halbgruppe. `$(M, *)$` heißt Monoid `$:\Leftrightarrow (M, *)$` hat ein neutrales Element `$e_M$`

Ein Element $e_M \in M$ heißt genau dann neutral, wenn gilt: $\forall a \in M: e_M * a = a * e_M = a$

Beispiele:

Jede Gruppe ist auch ein Monoid.
`$(\mathbb{N}_{0}, +)$` ist mit 0 als Neutralelement ein Monoid, aber keine Gruppe.
`$(\mathbb{N}_{0}, \cdot)$` ist mit 1 als Neutralelement ein Monoid, aber keine Gruppe.
`$(\mathbb{Z}, \cdot)$` ist mit 1 als Neutralelement ein Monoid, aber keine Gruppe.

Gruppe

Sei `$(M, *)$` ein Monoid. `$(M, *)$` heißt Gruppe `$:\Leftrightarrow \forall a \in M: \exists a^{-1} \in M: a * a^{-1} = a^{-1} * a = e_M$`

Beispiele:

Jeder Ring `$(R, +, \cdot)$` beinhaltet eine Gruppe `$(R, +)$`.
`$(\mathbb{Q}, +)$`
`$(\mathbb{R}, +)$`
`$(\mathbb{Q} \setminus \{0\}, \cdot)$`
`$(\mathbb{R} \setminus \{0\}, \cdot)$`

Ring

Sei `$R$` eine Menge und `$+:R \times R \rightarrow R$`, `$\cdot:R \times R \rightarrow R$` Verknüpfungen darauf. `$(R, +, \cdot)$` heißt Ring `$:\Leftrightarrow$`

`$(R,+)$` ist eine kommutative Gruppe (das Neutralelement heiße 0),
`$(R,\cdot)$` ist eine Halbgruppe und
die Distributivgesetze sind erfüllt.

Die Distributivgesetze lauten:

$\forall a,b,c,d \in R: (a+b) \cdot c = ac + bc$

und

$\forall a,b,c,d \in R: a \cdot (c+d) = ac + ad$

Außerdem:

Sei `$(R,+,\cdot)$` ein Ring. `$(R,+,\cdot)$` heißt Ring mit Eins `$:\Leftrightarrow (R, \cdot)$` ist Monoid.

Sei `$(R,+,\cdot)$` ein Ring. `$(R,+,\cdot)$` heißt kommutativer Ring `$:\Leftrightarrow (R, \cdot)$` ist kommutativ.

Beispiele:

Jeder Körper ist auch ein Ring.
`$(\mathbb{Z}, +, \cdot)$` ist ein kommutativer Ring mit Eins, aber kein Körper. Es fehlen die Inversen bei der Multiplikation.

Körper

Sei `$(K, +, \cdot)$` ein kommutativer Ring mit Eins. `$(K, +, \cdot)$` heißt Körper `$:\Leftrightarrow$`

`$(K,+)$` ist kommutativ.
`$(K \setminus \{0\}, \cdot)$` ist kommutative Gruppe.

Beispiele:

`$(\mathbb{Q}, +, \cdot)$`
`$(\mathbb{R}, +, \cdot)$`
`$(\mathbb{C}, +, \cdot)$`
`$(\mathbb{Z} / p \mathbb{Z}, +, \cdot)$`, wobei `$p$` eine Primzahl ist

Modul

Sei `$(R, +_R, \cdot_R)$` ein Ring und `$(M, +_M)$` eine kommutative Gruppe. Außerdem sei `$\cdot_V: R \times M \rightarrow M$` eine Abbildung. `$(M, R, \cdot_V)$` heißt R-Modul `$:\Leftrightarrow \forall \lambda, \mu \in R \; x,y \in M:$`

`$1_R \cdot x = x$`
`$\lambda \cdot (\mu \cdot x) = (\lambda \cdot \mu) \cdot x$`
`$(\lambda + \mu) \cdot x = \lambda \cdot x + \mu \cdot x$`
`$\lambda \cdot (x+y) = \lambda \cdot x + \lambda \cdot y$`

Beispiele:

Jeder K-Vektorraum ist auch ein K-Modul.
Das `$\mathbb{Z}$`-Modul `$\mathbb{Z}/2 \mathbb{Z}$` ist ein Modul ohne Basis, also kein Vektorraum.

Vektorraum

Sei `$(K, +_K, \cdot_K)$` ein Körper und `$(V,+_V)$` eine kommutative Gruppe. Außerdem sei `$\cdot_V: K \times V \rightarrow V$` eine skalalre Multiplikation. `$(V, K, \cdot_V)$` heißt `$K$`-Vektorraum `$:\Leftrightarrow \forall \lambda, \mu \in K \; x,y \in V:$`

`$1_K \cdot x = x$`
`$\lambda \cdot (\mu \cdot x) = (\lambda \cdot \mu) \cdot x$`
`$(\lambda + \mu) \cdot x = \lambda \cdot x + \mu \cdot x$`
`$\lambda \cdot (x+y) = \lambda \cdot x + \lambda \cdot y$`

Beispiele:

`$(\mathbb{R}[X], \mathbb{R}, \cdot_V)$`: Der Vektorraum der polynome mit Koeffizienten aus `$\mathbb{R}$`.

Weitere

Ideal

Ein Ideal in einem Ring `$(R, +, \cdot)$` ist eine Teilmenge `$I \subseteq R$`, die bezüglich der Addition eine Untergruppe ist und die folgende Eigenschaft hat: `$\forall x \in I, r \in R: xr \in I \text{ und } rx \in I$`

Beispiele:

Kerne von Ringhomomorphismen sind immer Ideale. (Und Ideale sind Kerne von Ringhomomorphismen.)

Algebra

Es sei `$R$` ein Ring. Eine `$R$`-Algebra ist ein Ring `$A$` zusammen mit einem Ringhomomorphismus `$\sigma: R \rightarrow A$`, sodass gilt: `$\forall r \in R, a \in A: \sigma(r) \cdot a = a \cdot \sigma(r)$`

Beispiele:

Jeder Ring ist eine `$\mathbb{Z}$`-Algebra.

Integritätsring

Es sei `$R$` ein vom Null-Ring verschiedener Ring. `$R$` heißt integritätsring `$:\Leftrightarrow R$` ist kommuativ und Nullteilerfrei.

Beispiele:

Zu dem Ring `$(\mathbb{Z}, +, \cdot)$` ist `$(\mathbb{Q}, +, \cdot)$` ein Quotientenkörper.

Hauptidealring

Es sei `$R$` ein Integritätsring. `$R$` heißt Hauptidealring `$:\Leftrightarrow$` Jedes Ideal `$I \subseteq R$` ist ein Hauptideal.

Quotientenkörper

Es sei `$R$` ein Integritätsring. Der kleinste Körper, in den `$R$` eingebettet werden kann, wird der Quotientenkörper des Integritätsrings genannt.

Beispiele:

Jeder Körper ist ein Integritätsring.
`$(\mathbb{Z}, +, \cdot)$`

Hilfsbegriffe

Ideal

Sei `$(R, +, \cdot)$` ein Ring und `$I \subseteq R$`. `$I$` heißt Ideal `$:\Leftrightarrow$`

`$(I, +)$` ist eine Gruppe,
`$\forall r \in R \forall a \in I: r \cdot a \in I$` und
`$\forall r \in R \forall a \in I: a \cdot r \in I$`.

Beispiele:

Die Menge `$2\mathbb{Z}$` der geraden ganzen Zahlen ist ein Ideal im Ring`$(\mathbb{Z}, +, \cdot)$`.

Hauptideal

Sei `$(R, +, \cdot)$` ein Ring und `$I \subseteq R$` ein Ideal. `$I$` heißt Hauptideal `$:\Leftrightarrow I$` wird von einem Element erzeugt.

Beispiele:

`$n\mathbb{Z}$`

Primideal

Sei `$(R, +, \cdot)$` ein Ring und `$I \subsetneq R$` ein Ideal. `$I$` heißt Primideal in `$R :\Leftrightarrow \forall x, y \in R: xy \in I \Rightarrow x \in I \lor y \in I$`

Beispiele:

`$2\mathbb{Z}$` ist Primideal in `$(\mathbb{Z}, +, \cdot)$`

Maximales Ideal

Sei `$(R, +, \cdot)$` ein Ring und `$I \subsetneq R$` ein Ideal. `$I$` heißt maximales Ideal in `$R :\Leftrightarrow$` Es gibt kein Ideal `$J$`, für das gilt: `$I \subsetneq J \subsetneq R$`

Beispiele:

In `$(\mathbb{Z},+,\cdot)$` ist jedes Primideal maximal.

Structs in C++

info@martin-thoma.de (Martin Thoma) — Fri, 07 Jun 2013 16:25:18 +0200

I guess this article isn’t very interesting, except if you have VERY little experience with C / C++. I only give some complete code examples. If you want some text, you could read C++ Programming/Structures (a wikibook).

Point

Basic example

#include <iostream>

using namespace std;

struct Point {
    double x;
    double y;
};

int main() {
    Point a = {12, 34};
    printf("(%.2f|%.2f)\n", a.x, a.y);
    return 0;
}

Functions

#include <iostream>
#define ABS(a) (a < 0 ? -(a) : a)

using namespace std;

struct Point {
    double x;
    double y;
};

double getManhattanDistance(Point a, Point b) {
    return ABS(a.x-b.x) + ABS(a.y-b.y);
}

int main() {
    Point a = {1, 1};
    Point b = {3, -4};
    printf("(%.2f|%.2f)\n", a.x, a.y);
    printf("(%.2f|%.2f)\n", b.x, b.y);
    printf("Distance: %.2f\n", getManhattanDistance(a, b));
    return 0;
}

More stuff

Initialization

Point a = {} initializes all values of point to 0.

Constructors

You can write constructors for structs:

#include <iostream>
#define ABS(a) (a < 0 ? -(a) : a)

using namespace std;

struct Point {
    double x;
    double y;

    Point():x(2.0),y(5.0) {}
    Point(double a, double b):x(9.0),y(9.0) {x=a; y=b;}
};

double getManhattanDistance(Point a, Point b) {
    return ABS(a.x-b.x) + ABS(a.y-b.y);
}

int main() {
    Point a = {};
    Point b = {3, -4};
    Point c;
    printf("(%.2f|%.2f)\n", a.x, a.y);
    printf("(%.2f|%.2f)\n", b.x, b.y);
    printf("(%.2f|%.2f)\n", c.x, c.y);
    printf("Distance: %.2f\n", getManhattanDistance(a, b));
    return 0;
}

Which gives:

./struct-example.out 
(2.00|5.00)
(3.00|-4.00)
(2.00|5.00)
Distance: 10.00

Functions in structs

You can also add functions to structs:

#include <iostream>
#define ABS(a) (a < 0 ? -(a) : a)

using namespace std;

struct Point {
    double x;
    double y;

    Point():x(2.0),y(5.0) {}
    Point(double a, double b):x(9.0),y(9.0) {x=a; y=b;}

    double getZeroDist() {
        return ABS(x)+ABS(y);
    }
};

double getManhattanDistance(Point a, Point b) {
    return ABS(a.x-b.x) + ABS(a.y-b.y);
}

int main() {
    Point a = {3, -10};
    printf("(%.2f|%.2f)\n", a.x, a.y);
    printf("Distance: %.2f\n", a.getZeroDist());
    return 0;
}

Result:

./struct-example.out 
(3.00|-10.00)
Distance: 13.00

Calculate square roots

info@martin-thoma.de (Martin Thoma) — Thu, 06 Jun 2013 20:54:10 +0200

Suppose you have an equation like this:

$x^2 = a, \;\;\;\; a \in \mathbb{R}_{\geq 0}$

Your task is to compute the solution $x \in \mathbb{R}_{\geq 0}$ .

How do you solve this algorithmically?

Input and output

Write a program that takes two parameters $a, n \in \mathbb{N}_{\geq 1}$ :

`$a$`: The number you should take the square root of. `$a$` is not bigger than 65535. To make this a little bit simpler, you can also assume that `$a$` is an integer.
`$n$`: Number of iterations you may use

Your program should output exactly one positive floating point number. The decimal separator is a point. At the end of the number should be a newline character \n.

Reference code

// Thanks to http://stackoverflow.com/a/15363123/562769
#include <stdio.h>
#include <gmp.h>

int main(int argc, char *argv[]) {
    mpf_t res, a;
    mpf_set_default_prec(1000000); // Increase this number.
    mpf_init(res);
    mpf_init(a);
    mpf_set_str(a, "2", 10);
    mpf_sqrt (res, a); 
    gmp_printf("%.1000Ff\n\n", res); // Increase this number.
    return 0;
}

You need GMP (libgmp-dev) to compile this.

Compile it like this:

gcc sqrt-reference.c  -lgmp -lm -O0 -g3 -o reference.out

This is the script I use to get the number of correct digits:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

def getScore(program, a, n):
    import os
    os.system("./reference.out " + str(a) + " > reference.txt")
    os.system("./" + program + " " + str(a) + " " + str(n) + " > result.txt")

    f = open('reference.txt', 'r')
    reference = f.read()
    f.close()

    f = open('result.txt', 'r')
    result = f.read()
    f.close()

    points = 0
    areEqual = True
    while reference[points] != "\n" and result[points] != "\n":
        if reference[points] == result[points]:
            points += 1
        else:
            break
    if points >= 2:
        points -= 1 # decimal point
    return points


if __name__ == "__main__":
    from argparse import ArgumentParser
     
    parser = ArgumentParser()
     
    # Add more options if you like
    parser.add_argument("-p", "--program", dest="program",
                      help="your program", metavar="FILE", required=True)
    parser.add_argument("-a", metavar='A', type=int, required=True,
                      help="calculate squre root of a")
    parser.add_argument("-n", metavar='N', type=int, required=True,
                      help="maximum n iterations")
     
    args = parser.parse_args()
 
print("Points for a=%i and n=%i: %i" % (args.a, args.n, getScore(args.program, args.a, args.n)))

Newton's method

How should be choose the initial value? I thought $\frac{a}{2}$ could be ok. In a good implementation you’ll probably do this with a lookup table.

With long double:

#include <iostream>
#include <cmath>
 
using namespace std;
 
long double newton(int a, int n) {
    long double x = ((long double)a)/2;
    for (int i=0; i<n; i++) {
        x = x - (x*x - a)/(2*x);
    }
    return x;
}
 
int main(int argc, char *argv[]) {
    if (argc != 3) {
        cout << "Please enter exactly two arguments." << endl;
        return 1;
    }
    int a = atoi(argv[1]);
    int n = atoi(argv[2]);
    printf("%.80Lf\n", newton(a, n));
    return 0;
}

I failed to convert this to a version that uses GMP :-/

But it converges quite fast:

Newtons method for calculating square roots

Exponential identity

According to Wikipedia (Source: Methods of computing square roots) many calculators use the following identity:

$\displaystyle \sqrt{a} = e^{\frac{1}{2}\ln a}$

But to calculate this, you need to be able to calculate $e^x$ for $x \in \mathbb{R}_{\geq 0}$ and $\ln(a)$ for $a \in \mathbb{R}_{\geq 0}$ .

I’ve used the definition of $e^x$ to calculate $e^x$ : $\displaystyle e^x := \sum_{i = 0}^\infty \frac{x^i}{i!}$

and:

$\displaystyle \ln(1+x) =- \sum_{k=0}^\infty \frac{(-x)^{k+1}}{k+1}$

So I gave it a try:

#include <iostream>
#include <cmath>
 
using namespace std;
 
long double ln(int S, int n) {
    long double tmp = S - 1;
    long double result = tmp;
    long double sign = 1.0;
    for (int i=2; i<n/2; i++) {
        tmp *= (S-1);
        sign *= -1;
        result += sign*tmp/i;
    }
    return result;
}

long double e(long double x, int n) {
    long double numerator = 1;
    long double denominator = 1;
    long double result = 1;
    for (int i=1; i<n/2; i++) {
        numerator *= x;
        denominator *= i;
        result += numerator/denominator;
    }
    return result;
}

long double sqrt(int a, int n) {
    return e(ln(a, n)*0.5, n);
}
 
int main(int argc, char *argv[]) {
    if (argc != 3) {
        cout << "Please enter exactly two arguments." << endl;
        return 1;
    }
    int a = atoi(argv[1]);
    int n = atoi(argv[2]);
    printf("%.80Lf\n", sqrt(a, n));
    return 0;
}

This converges VERY slow: For $a = 2$

`$n=1$`: 1 digit correct
`$n=10$`: 1 digit correct
`$n=100$`: 2 digits correct
`$n=1000$`: 4 digit correct
`$n=10,000$`: 5 digit correct
`$n=100,000$`: 5 digit correct
`$n=1,000,000$`: 6 digit correct

Ok, lets take a better Taylor series for calculating $e^x$ : $\displaystyle e^x = \sum_{k=0}^\infty \frac{x^{-1+2 k} (2 \cdot k+x)}{(2\cdot k)!}$

This didn’t change anything! I’m very surprised … as I’ve calculated $\pi$ for another article, changing the series to something similar improved the speed of convergence drastically.

Inverting matrices

info@martin-thoma.de (Martin Thoma) — Sun, 02 Jun 2013 14:01:57 +0200

Suppose you have a matrix $A \in \mathbb{R}^{n \times n}$ and you want to invert it. I’ve already explained how to invert a matrix (English explanation), but I didn’t provide any code and / or runtime analysis.

C++ Code

#include <iostream>
#include <cmath>
#include <vector>

using namespace std;

void print(vector< vector<double> > A) {
    int n = A.size();
    for (int i=0; i<n; i++) {
        for (int j=0; j<2*n; j++) {
            cout << A[i][j] << "\t";
            if (j == n-1) {
                cout << "| ";
            } 
        }
        cout << "\n";
    }
    cout << endl;
}

void calculateInverse(vector< vector<double> >& A) {
    int n = A.size();

    for (int i=0; i<n; i++) {
        // Search for maximum in this column
        double maxEl = abs(A[i][i]);
        int maxRow = i;
        for (int k=i+1; k<n; k++) {
            if (abs(A[k][i]) > maxEl) {
                maxEl = A[k][i];
                maxRow = k;
            }
        }

        // Swap maximum row with current row (column by column)
        for (int k=i; k<2*n;k++) {
            double tmp = A[maxRow][k];
            A[maxRow][k] = A[i][k];
            A[i][k] = tmp;
        }

        // Make all rows below this one 0 in current column
        for (int k=i+1; k<n; k++) {
            double c = -A[k][i]/A[i][i];
            for (int j=i; j<2*n; j++) {
                if (i==j) {
                    A[k][j] = 0;
                } else {
                    A[k][j] += c * A[i][j];
                }
            }
        }
    }

    // Solve equation Ax=b for an upper triangular matrix A
    for (int i=n-1; i>=0; i--) {
        for (int k=n; k<2*n;k++) {
            A[i][k] /= A[i][i];
        }
        // this is not necessary, but the output looks nicer:
        A[i][i] = 1; 

        for (int rowModify=i-1;rowModify>=0; rowModify--) {
            for (int columModify=n;columModify<2*n;columModify++) {
                A[rowModify][columModify] -= A[i][columModify] 
                                             * A[rowModify][i];
            }
            // this is not necessary, but the output looks nicer:
            A[rowModify][i] = 0;
        }
    }
}

int main() {
    int n;
    cin >> n;

    vector<double> line(2*n,0);
    vector< vector<double> > A(n,line);

    // Read input data
    for (int i=0; i<n; i++) {
        for (int j=0; j<n; j++) {
            cin >> A[i][j];
        }
    }

    for (int i=0; i<n; i++) {
        A[i][n+i] = 1;
    }

    // Print input
    print(A);

    // Calculate solution
    calculateInverse(A);

    // Print result
    cout << "Result:" << endl;
    print(A);
}

This code is VERY similar to the code of Gaussian elimination. In fact, I’ve only changed all occurrences of n+1 to 2*n. I also had to change lines 57-70, as we need to do all operations now on a matrix instead of a vector.

Time complexity

Lines 43-52:

$\displaystyle Operations = \sum_{i=0}^{n-1} \left (\sum_{k=i+1}^{n-1} (\sum_{j=i}^{2n} 1) \right ) = \frac{5}{6} n^3 - \frac{5}{6} n$ (see Wolfram|Alpha)

Lines 63-70:

$\displaystyle Operations = \sum_{i=0}^{n-1} \left (\sum_{k=0}^{i-1} (\sum_{j=n}^{2n} 1) \right ) = \frac{1}{2} n^3 - \frac{1}{2} n$ (see <a href=”http://www.wolframalpha.com/input/?i=sum_%7Bi%3D0%7D%5E%7Bn-1%7D+%28sum_%7Bk%3D0%7D%5E%7Bi-1%7D+%28sum_%7Bj%3Dn%7D%5E%7B2n%7D+1%29%29”))”>Wolfram Alpha</a>)

So we need about $\frac{4}{3} n^3 + \mathcal{O}(n^2)$ operations to invert a matrix with Gauß-Elimination.

Space complexity

My algorithm needs space for the inverse matrix, so it is in $\mathcal{O}(n^2)$ .

Inverting an upper triangular matrix

Suppose you have an upper triangular matrix $A \in \mathbb{R^{n \times n}}$ that you would like to invert. It could look like this:

$\begin{pmatrix} 2 & 7 & 1 & 8 & 2\\ 0 & 8 & 1 & 8 & 2\\ 0 & 0 & 8 & 4 & 5\\ 0 & 0 & 0 & 9 & 0\\ 0 & 0 & 0 & 0 & 4 \end{pmatrix}$

How could we improve the algorithm from above to get speed it up?

Well, we don’t need lines 24-53 any more, as those lines bring $A$ to an upper triangular form. But we still need lines 63-70. So we can improve the algorithm to a complexity of $\frac{1}{2} n^3 + \mathcal{O}(n^2)$ .

Fractions in C++

info@martin-thoma.de (Martin Thoma) — Sat, 01 Jun 2013 12:00:34 +0200

Today, I thought I should try to implement a class in C++ that deals with fractions. This is actually quite easy as I’ll show you.

First some math

Names

When you have a fraction $\frac{a}{b}$ then $a$ is called numerator and $b$ is called denominator.

Operations

The rules for basic operations are simple:

Addition: $\frac{a}{b} + \frac{c}{d} = \frac{a \cdot d + c \cdot b}{b \cdot d}$

Subtraction: $\frac{a}{b} - \frac{c}{d} = \frac{a \cdot d - c \cdot b}{b \cdot d}$

Multiplication: $\frac{a}{b} \cdot \frac{c}{d} = \frac{a \cdot c}{b \cdot d}$

Division: $\frac{a}{b} : \frac{c}{d} = \frac{a \cdot d}{b \cdot c}$

Euklids algorithm

You can calculate the greatest common divisor with Euklids algorithm. If you don’t know it, please read the Wikipedia article.

Knowing the greatest common divisor is important, because we want that our Faction class automatically cancels those factors so that the numerator and denominator are as small as possible.

C++ Code

#include <iostream>

using namespace std;

class Fraction {
    private:
        // Calculates the greates common divisor with 
        // Euclid's algorithm
        // both arguments have to be positive
        long long gcd(long long a, long long b) {
            while (a != b) {
                if (a > b) {
                    a -= b;
                } else {
                    b -= a;
                }
            }
            return a;
        }

    public:
        long long numerator, denominator;

        Fraction() {
            numerator = 0;
            denominator = 1;
        }

        Fraction(long long n, long long d) {
            if (d==0) {
                cerr << "Denominator may not be 0." << endl;
                exit(0);
            } else if (n == 0) {
                numerator = 0;
                denominator = 1;
            } else {
                int sign = 1;
                if (n < 0) {
                    sign *= -1;
                    n *= -1;
                }
                if (d < 0) {
                    sign *= -1;
                    d *= -1;
                }

                long long tmp = gcd(n, d);
                numerator = n/tmp*sign;
                denominator = d/tmp;
            }
        }

        operator int() {return (numerator)/denominator;}
        operator float() {return ((float)numerator)/denominator;}
        operator double() {return ((double)numerator)/denominator;}
};

Fraction operator+(const Fraction& lhs, const Fraction& rhs) {
    Fraction tmp(lhs.numerator*rhs.denominator
                +rhs.numerator*lhs.denominator,
                lhs.denominator*rhs.denominator);
    return tmp;
}

Fraction operator+=(Fraction& lhs, const Fraction& rhs) {
    Fraction tmp(lhs.numerator*rhs.denominator
                +rhs.numerator*lhs.denominator,
                lhs.denominator*rhs.denominator);
    lhs = tmp;
    return lhs;
}

Fraction operator-(const Fraction& lhs, const Fraction& rhs) {
    Fraction tmp(lhs.numerator*rhs.denominator
                -rhs.numerator*lhs.denominator,
                lhs.denominator*rhs.denominator);
    return tmp;
}

Fraction operator-=(Fraction& lhs, const Fraction& rhs) {
    Fraction tmp(lhs.numerator*rhs.denominator
                -rhs.numerator*lhs.denominator,
                lhs.denominator*rhs.denominator);
    lhs = tmp;
    return lhs;
}

Fraction operator*(const Fraction& lhs, const Fraction& rhs) {
    Fraction tmp(lhs.numerator*rhs.numerator,
               lhs.denominator*rhs.denominator);
    return tmp;
}

Fraction operator*=(Fraction& lhs, const Fraction& rhs) {
    Fraction tmp(lhs.numerator*rhs.numerator,
               lhs.denominator*rhs.denominator);
    lhs = tmp;
    return lhs;
}

Fraction operator*(int lhs, const Fraction& rhs) {
    Fraction tmp(lhs*rhs.numerator,rhs.denominator);
    return tmp;
}

Fraction operator*(const Fraction& rhs, int lhs) {
    Fraction tmp(lhs*rhs.numerator,rhs.denominator);
    return tmp;
}

Fraction operator/(const Fraction& lhs, const Fraction& rhs) {
    Fraction tmp(lhs.numerator*rhs.denominator,
                 lhs.denominator*rhs.numerator);
    return tmp;
}

std::ostream& operator<<(std::ostream &strm, const Fraction &a) {
    if (a.denominator == 1) {
        strm << a.numerator;
    } else {
        strm << a.numerator << "/" << a.denominator;
    }
    return strm;
}

int main() {
    Fraction a(1,3);
    Fraction b(3,28);
    Fraction c;

    c = a + b;
    cout << c << "\t(should be 37/84)" << endl;

    c = a - b;
    cout << c << "\t(should be 19/84)" << endl;

    c = a * b;
    cout << c << "\t(should be 1/28)" << endl;

    c = a / b;
    cout << c << "\t(should be 28/9)" << endl;

    c = -1 * b;
    cout << c << "\t(should be -3/28)" << endl;

    c = b * (-1);
    cout << c << "\t(should be -3/28)" << endl;

    c = Fraction(-100,3);
    cout <<    (int)c << "\t(should be -33)" << endl;
    cout <<  (float)c << "\t(should be -33.3...)" << endl;
    cout << (double)c << "\t(should be -33.3...)" << endl;

    a -= b;
    cout << a << "\t(should be 19/84)" << endl;

    return 0;
}

Solving linear equations with Gaussian elimination

info@martin-thoma.de (Martin Thoma) — Sun, 26 May 2013 00:01:30 +0200

Please note that you should use LU-decomposition to solve linear equations. The following code produces valid solutions, but when your vector \$b\$ changes you have to do all the work again. LU-decomposition is faster in those cases and not slower in case you don't have to solve equations with the same matrix twice.

Suppose you have a system of $n \in \mathbb{N_{\geq 1}}$ linear equations and variables $x_1, x_2, \dots, x_n \in \mathbb{R}$:

\[ \begin{align} a_{1,1} \cdot x_1 + a_{1,2} x_2 + \dots + a_{1,n} \cdot x_{n} &= b_1\\ a_{2,1} \cdot x_1 + a_{2,2} x_2 + \dots + a_{2,n} \cdot x_{n} &= b_2\\ \vdots &= \vdots\\ a_{n,1} \cdot x_1 + a_{n,2} x_2 + \dots + a_{n,n} \cdot x_{n} &= b_n \end{align}\]

All factors $a_{i,j} \in \mathbb{R}$ for $i,j \in 1, \dots, n$ can be written in one matrix $A \in \mathbb{R}^{n \times n}$ and all $b_i$ can be written as a vector $b$. You combine all $x_i$ in the same way to a vector $x$.

So you can write the system of equations as:

$A \cdot x = b$

How Gaussian elimination works

First, you write $A$ and (b) in an augmented matrix $(A|b)$:

\[ \left(\begin{array}{cccc|c} a_{1,1} & a_{1,2} & \dots & a_{1,n} & b_1\\ a_{2,1} & a_{2,2} & \dots & a_{2,n} & b_2\\ \vdots & \vdots & \ddots & \vdots & \vdots \\ a_{n,1} & a_{n,2} & \dots & a_{n,n} & b_n \end{array}\right)\]

On this matrix you may make exactly three operations:

Swap rows
Add one row onto another
Multiply every factor of one row with a constant

You want to get a triangular matrix. So you subsequently eliminate one variable from the system of equations until you have a matrix like this:

\[ \left(\begin{array}{ccccc|c} a_{1,1} & a_{1,2} & a_{1,3} & \dots & a_{1,n} & b_1\\ 0 & a_{2,2} & a_{2,3} & \dots & a_{2,n} & b_2\\ 0 & 0 & a_{3,3} & \dots & a_{3,n} & b_3\\ \vdots & \vdots & \ddots & \ddots& \vdots & \vdots\\ 0 & 0 & \dots & 0 & a_{3,n} & b_n\\ \end{array}\right)\]

It’s actually quite simple to get this form:

Pseudocode for Gaussian elimination

C++ Code

#include <iostream>
#include <cmath>
#include <vector>

using namespace std;

void print(vector< vector<double> > A) {
    int n = A.size();
    for (int i=0; i<n; i++) {
        for (int j=0; j<n+1; j++) {
            cout << A[i][j] << "\t";
            if (j == n-1) {
                cout << "| ";
            }
        }
        cout << "\n";
    }
    cout << endl;
}

vector<double> gauss(vector< vector<double> > A) {
    int n = A.size();

    for (int i=0; i<n; i++) {
        // Search for maximum in this column
        double maxEl = abs(A[i][i]);
        int maxRow = i;
        for (int k=i+1; k<n; k++) {
            if (abs(A[k][i]) > maxEl) {
                maxEl = abs(A[k][i]);
                maxRow = k;
            }
        }

        // Swap maximum row with current row (column by column)
        for (int k=i; k<n+1;k++) {
            double tmp = A[maxRow][k];
            A[maxRow][k] = A[i][k];
            A[i][k] = tmp;
        }

        // Make all rows below this one 0 in current column
        for (int k=i+1; k<n; k++) {
            double c = -A[k][i]/A[i][i];
            for (int j=i; j<n+1; j++) {
                if (i==j) {
                    A[k][j] = 0;
                } else {
                    A[k][j] += c * A[i][j];
                }
            }
        }
    }

    // Solve equation Ax=b for an upper triangular matrix A
    vector<double> x(n);
    for (int i=n-1; i>=0; i--) {
        x[i] = A[i][n]/A[i][i];
        for (int k=i-1;k>=0; k--) {
            A[k][n] -= A[k][i] * x[i];
        }
    }
    return x;
}

int main() {
    int n;
    cin >> n;

    vector<double> line(n+1,0);
    vector< vector<double> > A(n,line);

    // Read input data
    for (int i=0; i<n; i++) {
        for (int j=0; j<n; j++) {
            cin >> A[i][j];
        }
    }

    for (int i=0; i<n; i++) {
        cin >> A[i][n];
    }

    // Print input
    print(A);

    // Calculate solution
    vector<double> x(n);
    x = gauss(A);

    // Print result
    cout << "Result:\t";
    for (int i=0; i<n; i++) {
        cout << x[i] << " ";
    }
    cout << endl;
}

You can call it like this:

./gauss.out < 3x3.in
1        2        3        | 1
4        5        6        | 1
1        0        1        | 1

Result:        0 -1 1

Python code

#!/usr/bin/env python
# -*- coding: utf-8 -*-


def pprint(A):
    n = len(A)
    for i in range(0, n):
        line = ""
        for j in range(0, n+1):
            line += str(A[i][j]) + "\t"
            if j == n-1:
                line += "| "
        print(line)
    print("")


def gauss(A):
    n = len(A)

    for i in range(0, n):
        # Search for maximum in this column
        maxEl = abs(A[i][i])
        maxRow = i
        for k in range(i+1, n):
            if abs(A[k][i]) > maxEl:
                maxEl = abs(A[k][i])
                maxRow = k

        # Swap maximum row with current row (column by column)
        for k in range(i, n+1):
            tmp = A[maxRow][k]
            A[maxRow][k] = A[i][k]
            A[i][k] = tmp

        # Make all rows below this one 0 in current column
        for k in range(i+1, n):
            c = -A[k][i]/A[i][i]
            for j in range(i, n+1):
                if i == j:
                    A[k][j] = 0
                else:
                    A[k][j] += c * A[i][j]

    # Solve equation Ax=b for an upper triangular matrix A
    x = [0 for i in range(n)]
    for i in range(n-1, -1, -1):
        x[i] = A[i][n]/A[i][i]
        for k in range(i-1, -1, -1):
            A[k][n] -= A[k][i] * x[i]
    return x


if __name__ == "__main__":
    from fractions import Fraction
    n = input()

    A = [[0 for j in range(n+1)] for i in range(n)]

    # Read input data
    for i in range(0, n):
        line = map(Fraction, raw_input().split(" "))
        for j, el in enumerate(line):
            A[i][j] = el
    raw_input()

    line = raw_input().split(" ")
    lastLine = map(Fraction, line)
    for i in range(0, n):
        A[i][n] = lastLine[i]

    # Print input
    pprint(A)

    # Calculate solution
    x = gauss(A)

    # Print result
    line = "Result:\t"
    for i in range(0, n):
        line += str(x[i]) + "\t"
    print(line)

JavaScript code

/** Solve a linear system of equations given by a n&times;n matrix
    with a result vector n&times;1. */
function gauss(A) {
    var n = A.length;

    for (var i=0; i<n; i++) {
        // Search for maximum in this column
        var maxEl = Math.abs(A[i][i]);
        var maxRow = i;
        for(var k=i+1; k<n; k++) {
            if (Math.abs(A[k][i]) > maxEl) {
                maxEl = Math.abs(A[k][i]);
                maxRow = k;
            }
        }

        // Swap maximum row with current row (column by column)
        for (var k=i; k<n+1; k++) {
            var tmp = A[maxRow][k];
            A[maxRow][k] = A[i][k];
            A[i][k] = tmp;
        }

        // Make all rows below this one 0 in current column
        for (k=i+1; k<n; k++) {
            var c = -A[k][i]/A[i][i];
            for(var j=i; j<n+1; j++) {
                if (i==j) {
                    A[k][j] = 0;
                } else {
                    A[k][j] += c * A[i][j];
                }
            }
        }
    }

    // Solve equation Ax=b for an upper triangular matrix A
    var x= new Array(n);
    for (var i=n-1; i>-1; i--) {
        x[i] = A[i][n]/A[i][i];
        for (var k=i-1; k>-1; k--) {
            A[k][n] -= A[k][i] * x[i];
        }
    }
    return x;
}

PHP

<?php
/**
 * Gaussian elimination
 * @param  array $A matrix
 * @param  array $x vector
 * @return array    solution vector
 */
function gauss($A, $x) {
    # Just make a single matrix
    for ($i=0; $i < count($A); $i++) {
        $A[$i][] = $x[$i];
    }
    $n = count($A);

    for ($i=0; $i < $n; $i++) {
        # Search for maximum in this column
        $maxEl = abs($A[$i][$i]);
        $maxRow = $i;
        for ($k=$i+1; $k < $n; $k++) {
            if (abs($A[$k][$i]) > $maxEl) {
                $maxEl = abs($A[$k][$i]);
                $maxRow = $k;
            }
        }


        # Swap maximum row with current row (column by column)
        for ($k=$i; $k < $n+1; $k++) {
            $tmp = $A[$maxRow][$k];
            $A[$maxRow][$k] = $A[$i][$k];
            $A[$i][$k] = $tmp;
        }

        # Make all rows below this one 0 in current column
        for ($k=$i+1; $k < $n; $k++) {
            $c = -$A[$k][$i]/$A[$i][$i];
            for ($j=$i; $j < $n+1; $j++) {
                if ($i==$j) {
                    $A[$k][$j] = 0;
                } else {
                    $A[$k][$j] += $c * $A[$i][$j];
                }
            }
        }
    }

    # Solve equation Ax=b for an upper triangular matrix $A
    $x = array_fill(0, $n, 0);
    for ($i=$n-1; $i > -1; $i--) {
        $x[$i] = $A[$i][$n]/$A[$i][$i];
        for ($k=$i-1; $k > -1; $k--) {
            $A[$k][$n] -= $A[$k][$i] * $x[$i];
        }
    }

    return $x;
}
?>

And some tiny tests:

<?php
$A = array(array(7));
$x = array(3);
$result = gauss($A, $x);
var_dump($result);
### array (size=1)
###      0 => float 0.42857142857143 = 3 / 7

$A = array(array(7));
$x = array(0);
$result = gauss($A, $x);
var_dump($result);
### array (size=1)
###      0 => int 0
?>

Test equations with two variables (see [Wolfram

Alpha](http://www.wolframalpha.com/input/?i=solve+%7B%7B7%2C+1%7D%2C+%7B5%2C+3%7D%7D+*+%7B%7Bx%7D%2C+%7By%7D%7D+%3D+%7B%7B1%7D%2C+%7B3%7D%7D)):

<?php
$A = array(array(7, 1), array(5, 3));
$x = array(1, 3);
$result = gauss($A, $x);
var_dump($result);
### array (size=2)
###  0 => float 0
###  1 => float 1
?>

Test equations with two variables (see [Wolfram

Alpha](http://www.wolframalpha.com/input/?i=solve+%7B%7B7%2C+1%7D%2C+%7B5%2C+3%7D%7D+*+%7B%7Bx%7D%2C+%7By%7D%7D+%3D+%7B%7B1%7D%2C+%7B1%7D%7D)):

<?php
$A = array(array(7, 1), array(5, 3));
$x = array(1, 1);
$result = gauss($A, $x);
var_dump($result);
### array (size=2)
###  0 => float 0.125
###  1 => float 0.125
?>

Complexity

Time complexity

Time complexity is in $\mathcal{O}(n^3)$ (lines 44 - 53):

\[\begin{align} Operations &= \sum_{i=0}^{n-1} \sum_{k=i+1}^{n-1} \sum_{j=i}^{n} 1\\ &= \sum_{i=0}^{n-1} \sum_{k=i+1}^{n-1} (n-i+1) \\ &= \left (\sum_{i=0}^{n-1} \sum_{k=i+1}^{n-1} (n+1) \right ) - \left (\sum_{i=0}^{n-1} \sum_{k=i+1}^{n-1} i \right )\\ &= \dots \\ &= \frac{1}{6} \cdot n \cdot (2 n^2+3 n-5)\\ &= \frac{1}{3} \cdot n^3 + \mathcal{O}(n^2) \end{align}\]

Space complexity

Space complexity of this implementation is in $\mathcal{O}(n)$, but you can easily come down to $\mathcal{O}(1)$ when you use A[n] for storing x.

Generating many prime numbers

info@martin-thoma.de (Martin Thoma) — Tue, 21 May 2013 22:50:10 +0200

Today, a fellow student claimed that it would take much time to check the first 1,000,000 numbers for primes. I claimed that it would be a matter of seconds to do so for the first 1,000,000,000 numbers.

So, lets prove my claim.

Trivial approach

#include <iostream> // cin, cout
#include <cmath> // sqrt
#include <vector>
 
using namespace std;
vector<unsigned long> primeList;

bool isPrime(unsigned long n) {
    vector<unsigned long>::iterator myIt;
    unsigned long root = (unsigned long) sqrt(n);
    for(myIt=primeList.begin(); myIt != primeList.end(); myIt++){
        if(n%(*myIt) == 0) {
            return false;
        } else if ((*myIt) > root) {
            return true;
        }
    }
    return true;
}

int main(int argc, char* argv[]) {
    primeList.push_back(2);
    cout << 2 << endl;

    unsigned long long max = (unsigned long long) atoi(argv[1]);
    for (unsigned long i=3; i<max; i+=2) {
        if(isPrime(i)) {
            primeList.push_back(i);
            cout << i << endl;
        }
    }

    return 0;
}

Execute it for 100,000,000:

time ./generate-prime-list.out 100000000 > primes.txt

real	0m57.274s
user	0m41.855s
sys	0m15.229s

So 41 seconds for all primes not bigger than 100,000,000.

Now, lets test it for 1,000,000,000:

time ./generate-prime-list.out 1000000000 > primes.txt

real	18m18.205s
user	15m56.904s
sys	2m12.500s

16 minutes … not exactly “seconds”. But please keep in mind that I also wrote the result to a txt-file. This txt file is 502.0 MB big. It takes quite a lot of time to write such an amount of data from memory to disk.

Sieve of Eratosthenes

A first try

#include <iostream> // cin, cout
#include <vector>
 
using namespace std;

void sieveOfEratosthenes(unsigned long n) {
    vector<bool> primesEratosthenes (n+1, true);
    cout << 2 << endl;
    for (unsigned long i=3; i<n; i+=2) {
        if (primesEratosthenes[i] == true) {
            cout << i << endl;
            for (unsigned long j=2; j*i<=n; j++) {
                primesEratosthenes[j*i] = false;
            }
        }
    }
}

int main(int argc, char* argv[]) {
    unsigned long long n = (unsigned long long) atoi(argv[1]);
    sieveOfEratosthenes(n);
    return 0;
}

This one takes 1 minute 5 seconds:

make;time ./generate-prime-list.out 1000000000 > testPrimes.txt
g++ -std=c++0x -Wall -pedantic -O3 -D NDEBUG generate-prime-list.cpp -o generate-prime-list.out

real	3m20.436s
user	1m4.908s
sys	2m11.748s

I’m getting closer to “seconds”. ☺

ofstream, endl, \n and buffers

Writing 502.0 MB takes some time. It’s not getting better when I pipe this through bash. So lets write it directly:

#include <fstream> // ofstream
#include <vector>
 
using namespace std;

void sieveOfEratosthenes(unsigned long n) {
    ofstream myfile;
    myfile.open ("huge-prime-list.txt");
    vector<bool> primesEratosthenes (n+1, true);
    myfile << 2 << endl;
    for (unsigned long i=3; i<n; i+=2) {
        if (primesEratosthenes[i] == true) {
            myfile << i << endl;
            for (unsigned long j=2; j*i<=n; j++) {
                primesEratosthenes[j*i] = false;
            }
        }
    }
    myfile.close();
}

int main(int argc, char* argv[]) {
    unsigned long long n = (unsigned long long) atoi(argv[1]);
    sieveOfEratosthenes(n);
    return 0;
}

time ./generate-prime-list.out 1000000000

real	3m21.249s
user	1m4.016s
sys	2m12.332s

Ok, no real change :-/

Another idea is to replace endl by \n (see explanation)

That was a good try. Now it needs only 46 seconds:

time ./generate-prime-list.out 1000000000

real	0m49.539s
user	0m46.619s
sys	0m0.920s

Another reason why this might be slow could be too many system calls. So I could buffer some and write them in blocks. I could also just write blocks of unsigned long numbers instead of a string representation. This might lead to a much smaller file size which in consequence is faster to write:

#include <stdio.h> // fopen
#include <iostream> // atoi
#include <vector>

using namespace std;

void sieveOfEratosthenes(unsigned long n) {
    FILE* pFile;
    pFile = fopen("huge-prime-list.txt", "wb");
    vector<bool> primesEratosthenes (n+1, true);

    unsigned long tmp = 2;
    fwrite(&tmp, sizeof(unsigned long),1, pFile);
    for (unsigned long i=3; i<n; i+=2) {
        if (primesEratosthenes[i] == true) {
            fwrite(&i, sizeof(unsigned long),1, pFile);
     
            for (unsigned long j=2; j*i<=n; j++) {
                primesEratosthenes[j*i] = false;
            }
        }
    }

    fclose(pFile);
}

int main(int argc, char* argv[]) {
    unsigned long long n = (unsigned long long) atoi(argv[1]);
    sieveOfEratosthenes(n);
    return 0;
}

And execute it:

time ./generate-prime-list.out 1000000000

real	0m39.700s
user	0m38.546s
sys	0m0.640s

38 seconds for all primes from 2 to 1,000,000,000. The file size is now only 203.4 MB.

By the way, simply setting this with setbuf(pFile, NULL); to unbuffered resulted in 50 seconds execution time.

You can get the list with this snippet:

#include <iostream>
#include <fstream>

using namespace std;

int main(int argc, char* argv[]) {
    if (argc != 2) {
        cout << "You have to specify a file name" << endl;
    } else {
        FILE* pFile;
        pFile = fopen(argv[1], "rb");

        long long x;
        size_t read;
        while (!feof(pFile)) {
            read = fread(&x, sizeof(long long), 1, pFile);
            (void) read;
            if (feof(pFile)){
                break; // otherwise it duplicates the last entry
            }
            cout << x << endl;
        }
        fclose(pFile);
    }
}

Improve sieving

The following change was suggested by Niklas B.. Thanks!

Take a look at the inner for loop. This one does the sieving, so it gets executed very often. In this loop, you have to calculate j*i for checking the condition of the loop and again for setting it to false. You can get rid of one of those operations. Additionally, you don’t have to start sieving at 2*p, but you can start at p*p as you already sieved out all multiples of the first, second, …, current-1-th prime.

#include <stdio.h> // fopen
#include <iostream> // atoi
#include <vector>
 
using namespace std;
 
void sieveOfEratosthenes(long long n) {
    FILE* pFile;
    pFile = fopen("huge-prime-list.bin", "wb");
    vector<bool> primesEratosthenes (n+1, true);
 
    long long tmp = 2;
    fwrite(&tmp, sizeof(long long),1, pFile);
    for (long long i=3; i<n; i+=2) {
        if (primesEratosthenes[i]) {
            fwrite(&i, sizeof(long long), 1, pFile);
      
            for (long long j=i*i; j<=n; j+=i) {
                primesEratosthenes[j] = false;
            }
        }
    }
 
    fclose(pFile);
}
 
int main(int argc, char* argv[]) {
    if (argc != 2) {
        cout << "You have to specify n" << endl;
    } else {
        long long  n = (long long) atoi(argv[1]);
        sieveOfEratosthenes(n);
    }
    return 0;
}

Execute:

time ./generate-prime-list.out 1000000000

real	0m24.222s
user	0m23.485s
sys	0m0.436s

primesieve

Primesieve is a free software program and C++ library that generates prime numbers and prime k-tuplets (twin primes, prime triplets, …) $< 2^{64}$ using a highly optimized implementation of the sieve of Eratosthenes.

According to the GUI, it finds all 50,847,534 primes below 1,000,000,000 in 0.16 seconds. But write them to a file…

Sieve of Atkin

Arthur Oliver Lonsdale Atkin (July 31, 1925 – December 28, 2008) was a British mathematician who invented this sieve. I’ve implemented it according to the pseudocode provided in Wikipedia. A very long explanation of Atkins sieve is on StackOverflow

My first implementation

#include <stdio.h> // fopen
#include <iostream> // atoi
#include <vector>
#include <cmath> // sqrt, ceil, 

using namespace std;

void sieveOfAtkin(long long limit) {
    FILE* pFile;
    pFile = fopen("huge-prime-list.bin", "wb");

	long long root = ceil(sqrt(limit));

    // initialize the sieve
	vector<bool> is_prime(limit, false);

    // put in candidate primes: 
    // integers which have an odd number of
    // representations by certain quadratic forms
	for (long long x = 1; x <= root; x++) {
        long long xSquare = x*x;
		for (long long y = 1; y <= root; y++) {
			long long n = (4*xSquare)+(y*y);
			if (n <= limit && (n % 12 == 1 || n % 12 == 5)) {
                is_prime[n] = !is_prime[n];
            }

			n = (3*xSquare)+(y*y);
			if (n <= limit && n % 12 == 7) {
                is_prime[n] = !is_prime[n];
            }

			n = (3*xSquare)-(y*y);
			if (x > y && n <= limit && n % 12 == 11) {
                is_prime[n] = !is_prime[n];
            }
		}
	}

    // eliminate composites by sieving
	for (long long n = 5; n <= root; n++) {
        if (is_prime[n]) {
            long long add = n*n;
            for (long long k = add; k < limit; k += add) {
                // n is prime, omit multiples of its square; this is
                // sufficient because composites which managed to get
                // on the list cannot be square-free
                is_prime[k] = false;
            }
        }
    }

    // Output primes
    long long primTmp = 2;
    fwrite(&primTmp, sizeof(long long), 1, pFile);
    primTmp = 3;
    fwrite(&primTmp, sizeof(long long), 1, pFile);

	for (long long n = 5; n < limit; n++) {
		if (is_prime[n]) {
            fwrite(&n, sizeof(long long), 1, pFile);
		}
	}

    fclose(pFile);
}

int main(int argc, char* argv[]) {
    if (argc != 2) {
        cout << "You have to specify n" << endl;
    } else {
        long long  n = (long long) atoi(argv[1]);
        sieveOfAtkin(n);
    }
    return 0;
}

Atkins sieve has a much worse performance than Sieve of Eratosthenes:

time ./generate-prime-list.out 1000000000

real	1m6.001s
user	1m4.724s
sys	0m0.604s

Primegen

Primegen is an implementation by Daniel J. Bernstein. It’s a little bit cluttered with 80 files and 3854 LOC in total.

When I have some time, I’ll update this article and create a new version of my sieve with ideas from Primegen.

Primegen is fast:

time ./primes > primes.txt

real	0m11.677s
user	0m10.649s
sys	0m0.708s

About 11 seconds for writing all primes between 2 and 1,000,000,000 to a txt file.

Finally

The last script that took 23 seconds for calculating and writing all primes in 2 to 1,000,000,000 seems to be the best one. Do you know anything else that could get improved? Please provide a working example (e.g. as a gist)

Solving equations of upper triangular matrices

info@martin-thoma.de (Martin Thoma) — Mon, 20 May 2013 10:41:49 +0200

Suppose you have an equation like $R \cdot x = b$ with $R \in \mathbb{R}^{n \times n}$ and $x,b \in \mathbb{R}^n$. $b$ and $R$ are given and you want to solve for $x$.

Example

With $n=5$, the problem could look like this:

\[\begin{pmatrix} 2 & 7 & 1 & 8 & 2
0 & 8 & 1 & 8 & 2
0 & 0 & 8 & 4 & 5
0 & 0 & 0 & 9 & 0
0 & 0 & 0 & 0 & 4 \end{pmatrix} \cdot \begin{pmatrix} x_1 \ x_2 \ x_3 \ x_4 \ x_5 \end{pmatrix} = \begin{pmatrix} 3 \ 1 \ 4 \ 1 \ 5 \end{pmatrix}\]

This is only a shorthand for:

\[\begin{align} 2 \cdot x_1 + 7 \cdot x_2 + 1 \cdot x_3 + 8 \cdot x_4 + 2 \cdot x_5 &= 3
8 \cdot x_2 + 1 \cdot x_3 + 8 \cdot x_4 + 2 \cdot x_5 &= 1
8 \cdot x_3 + 4 \cdot x_4 + 5 \cdot x_5 &= 4
9 \cdot x_4 + 0 \cdot x_5 &= 1
4 \cdot x_5 &= 5 \end{align} \]

First step: Solve for \$x_5\$

First you see that $x_5 = \frac{5}{4}$. So you divide $b$ by the current row.

Don't divide through 0. When you would have to divide by 0 and b is 0, this system has an infinite amount of solutions. When you would have to divide by 0 and b is not 0, then this system has no solution.

Now you replace every occurrence of $x_5$ in the system of equations above:

\[\begin{align} 2 \cdot x_1 + 7 \cdot x_2 + 1 \cdot x_3 + 8 \cdot x_4 + 2 \cdot \frac{5}{4} &= 3
8 \cdot x_2 + 1 \cdot x_3 + 8 \cdot x_4 + 2 \cdot \frac{5}{4} &= 1
8 \cdot x_3 + 4 \cdot x_4 + 5 \cdot \frac{5}{4} &= 4
9 \cdot x_4 + 0 \cdot \frac{5}{4} &= 1
4 \cdot \frac{5}{4} &= 5 \end{align} \]

Now you make the multiplications and remove the first trivial line.

\[ \begin{align} 2 \cdot x_1 + 7 \cdot x_2 + 1 \cdot x_3 + 8 \cdot x_4 + \frac{5}{2} &= 3
8 \cdot x_2 + 1 \cdot x_3 + 8 \cdot x_4 + \frac{5}{2} &= 1
8 \cdot x_3 + 4 \cdot x_4 + \frac{25}{4} &= 4
9 \cdot x_4 + 0 &= 1
\end{align} \]

Second step: update

Get the constant factors to the right side of the equations: \[ \begin{align} 2 \cdot x_1 + 7 \cdot x_2 + 1 \cdot x_3 + 8 \cdot x_4 &= \frac{1}{2}
8 \cdot x_2 + 1 \cdot x_3 + 8 \cdot x_4 &= -\frac{3}{2}
8 \cdot x_3 + 4 \cdot x_4 &= -\frac{9}{4}
9 \cdot x_4 &= 1
\end{align} \]

You’re now in the same situation as in the first step. Next you will solve for $x_4$, then for $x_3, x_2$ and finally for $x_1$.

This is called “back substitution”.

According to Wolfram|Alpha, the solution is:

\[x = \frac{1}{4608} \cdot \begin{pmatrix}4017\-1182\-1552\512\5760\end{pmatrix} = \begin{pmatrix}\frac{1339}{1536} \ -\frac{197}{768} \ -\frac{97}{288} \ \frac{1}{9} \ \frac{5}{4}\end{pmatrix}\]

Python straightforward algorithm

I will use fractions for operations as I don’t want to lose precision while dividing.

#!/usr/bin/env python
# -*- coding: utf-8 -*-

def solveUpperTriangularMatrix(R, b):
    # Convert R and b to Fraction
    from fractions import Fraction
    fR, fb = [], []
    for x, line in enumerate(R):
        fLine = []
        for y, el in enumerate(line):
            fLine.append(Fraction(el))
        fR.append(fLine)
    for el in b:
        fb.append(Fraction(el))

    # The solution will be here
    x = [Fraction(0)] * len(b)
    for step in range(len(b)-1, 0-1, -1):
        if fR[step][step] == 0:
            if fb[step] != 0:
                return "No solution"
            else:
                return "Infinity solutions"
        else:
            x[step] = fb[step] / fR[step][step]

        for row in range(step-1, 0-1, -1):
            fb[row] -= fR[row][step]*x[step]
    return x

if __name__ == "__main__":
    R = [[2, 7, 1, 8, 2],
         [0, 8, 1, 8, 2],
         [0, 0, 8, 4, 5],
         [0, 0, 0, 9, 0],
         [0, 0, 0, 0, 4]]
    b =  [3, 1, 4, 1, 5]
    x = solveUpperTriangularMatrix(R, b)
    print(x)

    # Convert x to float
    x = map(float, x)
    print(x)

A better algorithm

Just like for unipotent lower triangular matrices, we can operate directly on the given input:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

def solveUpperTriangularMatrix(R, b):
    # Convert R and b to Fraction
    from fractions import Fraction
    for x, line in enumerate(R):
        for y, el in enumerate(line):
            R[x][y] = Fraction(el)
    for x, el in enumerate(b):
        b[x] = Fraction(el)

    # The solution will be here
    for step in range(len(b)-1, 0-1, -1):
        if R[step][step] == 0:
            if b[step] != 0:
                return "No solution"
            else:
                return "Infinity solutions"
        else:
            b[step] = b[step] / R[step][step]

        for row in range(step-1, 0-1, -1):
            b[row] -= R[row][step]*b[step]

if __name__ == "__main__":
    R = [[2, 7, 1, 8, 2],
         [0, 8, 1, 8, 2],
         [0, 0, 8, 4, 5],
         [0, 0, 0, 9, 0],
         [0, 0, 0, 0, 4]]
    b =  [3, 1, 4, 1, 5]
    solveUpperTriangularMatrix(R, b)
    print(b)

Conversion to Fraction

You could think that the conversion to fraction is not necessary. But if you simply remove line 5 to 16, you will get:

[1, 0, -1, 0, 1]

because of integer arithmetic. When you convert the input to float before passing it to solveUpperTriangularMatrix, you will get

[0.8717447916666666, -0.25651041666666663, -0.3368055555555556, 0.1111111111111111, 1.25]

which is almost the same as when we calculated with Fraction and converted to float afterwards:

[0.8717447916666666, -0.2565104166666667, -0.3368055555555556, 0.1111111111111111, 1.25]

So: Using Fractions needs some computing time, but you will get better results.

Time complexity

I’ll analyze the second algorithm.

The conversion of our input data is obviously in $\mathcal{O}(n^2)$. Let’s only analyse the part after the conversion.

Assume that there is exactly one solution and that line 15-21 take $c_1$ operations and line 24 takes $c_2$ operations.

Then we would have a total of

\[\begin{align} \text{Operations} &= \sum_{i=1}^n \left ( c_1 + \sum_{j=1}^{i-1} c_2 \right )
&= \sum_{i=1}^n c_1 + \sum_{i=1}^n \sum_{j=1}^{i-1} c_2
&= n \cdot c_1 + c_2 \cdot \left (\sum_{i=1}^n \sum_{j=1}^{i-1} 1 \right )
&= n \cdot c_1 + c_2 \cdot \left (\sum_{i=1}^n (i-1) \right )
&= n \cdot c_1 + c_2 \cdot \left ((\sum_{i=1}^n i) - (\sum_{i=1}^n 1) \right )
&= n \cdot c_1 + c_2 \cdot \left (\frac{n^2+n}{2} - n \right )
&= n \cdot c_1 + (n^2-n) \cdot \frac{c_2}{2}
\end{align}\]

So the algorithms time complexity is in $\Theta(n^2) \subsetneq \mathcal{O}(n^2)$.

Space complexity

Please note that I take advantage of Pythons dynamic typing system. I think it’s difficult to see space complexity in python programs. But when you make the same in C++, you will see that you will need space in $\mathcal{O}(n)$ when you do the conversion. Without the conversion, you’re in $\mathcal{O}(1)$.

I guess you might want to leave this choice to the user of your functions. When he wants better results, he should give the input as Fraction. When he wants to get results rather faster, he should give the input as float.

Notes

In this algorithm, we don’t need anything below the diagonal.

Solving equations of lower unitriangular matrices

info@martin-thoma.de (Martin Thoma) — Sun, 19 May 2013 22:15:59 +0200

Suppose you have an equation like $L \cdot x = b$ with $L \in \mathbb{R}^{n \times n}$ and $x,b \in \mathbb{R}^n$ . $b$ and $L$ are given and you want to solve for $x$ .

Example

With $n=5$ , the problem could look like this:

$\begin{pmatrix} 1 & 0 & 0 & 0 & 0\\ 2 & 1 & 0 & 0 & 0\\ 7 & 1 & 1 & 0 & 0\\ 8 & 2 & 8 & 1 & 0\\ 1 & 8 & 2 & 8 & 1 \end{pmatrix} \cdot \begin{pmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \\ x_5 \end{pmatrix} = \begin{pmatrix} 3 \\ 1 \\ 4 \\ 1 \\ 5 \end{pmatrix}$

This is only a shorthand for: $ \begin{align} &1 \cdot x_1 &= 3 \\ &2 \cdot x_1 + 1 \cdot x_2 &= 1\\ &7 \cdot x_1 + 1 \cdot x_2 + 1 \cdot x_3 &= 4\\ &8 \cdot x_1 + 2 \cdot x_2 + 8 \cdot x_3 + 1 \cdot x_4 &= 1\\ &1 \cdot x_1 + 8 \cdot x_2 + 2 \cdot x_3 + 8 \cdot x_4 + 1 \cdot x_5 &= 5 \end{align} $

This is easy to solve, isn’t it?

First step: Solve for `$x_1$`

First you see that $x_1 = 3$ . Now you replace every occurence of $x_1$ in the system of equations above:

$ \begin{align} &1 \cdot 3 &= 3 \\ &2 \cdot 3 + 1 \cdot x_2 &= 1\\ &7 \cdot 3 + 1 \cdot x_2 + 1 \cdot x_3 &= 4\\ &8 \cdot 3 + 2 \cdot x_2 + 8 \cdot x_3 + 1 \cdot x_4 &= 1\\ &1 \cdot 3 + 8 \cdot x_2 + 2 \cdot x_3 + 8 \cdot x_4 + 1 \cdot x_5 &= 5 \end{align}$

Now you make the multiplications and remove the first trivial line.

$ \begin{align} &6 + 1 \cdot x_2 &= 1\\ &21 + 1 \cdot x_2 + 1 \cdot x_3 &= 4\\ &24 + 2 \cdot x_2 + 8 \cdot x_3 + 1 \cdot x_4 &= 1\\ &3 + 8 \cdot x_2 + 2 \cdot x_3 + 8 \cdot x_4 + 1 \cdot x_5 &= 5 \end{align} $

Second step: update

Get the constant factors to the right side of the equations: $ \begin{align} &1 \cdot x_2 &= 1-6=-5\\ &1 \cdot x_2 + 1 \cdot x_3 &= 4-21=-17\\ &2 \cdot x_2 + 8 \cdot x_3 + 1 \cdot x_4 &= 1-24=-23\\ &8 \cdot x_2 + 2 \cdot x_3 + 8 \cdot x_4 + 1 \cdot x_5 &= 5-3=2 \end{align} $

You can now easily see that you’re in the same situation as in the first step! Next you will solve for $x_2$ , then for $x_3, x_4$ and finally for $x_5$ .

This is the reason why solving such a system of equations is sometimes called “forward substitution”.

Python straightforward algorithm

#!/usr/bin/env python
# -*- coding: utf-8 -*-

def solveLowerUnitriangularMatrix(L, b):
    x = [0] * len(b)
    for step in range(0, len(b)):
        x[step] = b[step]
        for row in range(0, len(b)):
            b[row] = b[row] - L[row][step]*x[step]
    return x

if __name__ == "__main__":
    L = [[1, 0, 0, 0, 0],
         [2, 1, 0, 0, 0],
         [7, 1, 1, 0, 0],
         [8, 2, 8, 1, 0],
         [1, 8, 2, 8, 1]]
    b =  [3, 1, 4, 1, 5]

    print(solveLowerUnitriangularMatrix(L, b))

Pretty easy, isn’t it? But can we even do better?

Even better algorithm

Yes, we can!

Take a look at what’s happening when row = 0 in line 9. We make a step that is not necessary. Also, we can take the space of b to store x!

#!/usr/bin/env python
# -*- coding: utf-8 -*-

def solveLowerUnitriangularMatrix(L, b):
    for step in range(0, len(b)):
        for row in range(step+1, len(b)):
            b[row] -= L[row][step]*b[step]

if __name__ == "__main__":
    L = [[1, 0, 0, 0, 0],
         [2, 1, 0, 0, 0],
         [7, 1, 1, 0, 0],
         [8, 2, 8, 1, 0],
         [1, 8, 2, 8, 1]]
    b =  [3, 1, 4, 1, 5]

    solveLowerUnitriangularMatrix(L, b)
    print(b)

Now it looks super clean, doesn’t it ☺

Keep in mind that you have to store b if you need the values after you’ve applied this algorithm. This is the reason why there here. This algorithm “returns” its value by manipulating b.

Time complexity

I’ll analyze the second algorithm.

Let’s assume that line 7 takes $c$ operations and $n$ is the size of $L \in \mathbb{R}^{n \times n}$ .

Then we would have a total of

$\begin{align} \text{Operations} &= \sum_{i=1}^n \left ( \sum_{j=i+1}^n c \right )\\ &= c \cdot \sum_{i=1}^n \left ( \sum_{j=i+1}^n 1 \right )\\ &= c \cdot \sum_{i=1}^n (n - i)\\ &= c \cdot \left ( \sum_{i=1}^n n - \sum_{i=1}^n i \right )\\ &= c \cdot \left ( n^2 - \frac{n^2+n}{2} \right )\\ &= \frac{c}{2} (n^2 - n) \end{align}$

So the algorithms time complexity is in $\Theta(n^2) \subsetneq \mathcal{O}(n^2)$ .

Space complexity

Well, thats simple: $\mathcal{O}(1)$ !

I do ignore the size of the input. So $\mathcal{O}(1)$ means: For variable sized input data I do need a constant amount of additional space.

More improvements

In the last algorithm I’ve presented you can see that we actually don’t check the values on or above of the diagonal. This means, the following two function calls do give the same b:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

def solveLowerUnitriangularMatrix(L, b):
    for step in range(0, len(b)):
        for row in range(step+1, len(b)):
            b[row] -= L[row][step]*b[step]
    return L

if __name__ == "__main__":
    L = [[1, 0, 0, 0, 0],
         [2, 1, 0, 0, 0],
         [7, 1, 1, 0, 0],
         [8, 2, 8, 1, 0],
         [1, 8, 2, 8, 1]]
    b =  [3, 1, 4, 1, 5]
    solveLowerUnitriangularMatrix(L, b)
    print(b)

    L = [[10, 9, 8, 7, 6],
         [2, 5, 4, 3, 2],
         [7, 1, 1, 0, 1],
         [8, 2, 8, 2, 3],
         [1, 8, 2, 8, 4]]
    b =  [3, 1, 4, 1, 5]
    solveLowerUnitriangularMatrix(L, b)
    print(b)

So, theoretically, we could store some other information on and above of the diagonal. We also don’t change L. Keep this in mind, this might be important in later articles.

k-nearest-neighbor classification and k-means - an interactive example

info@martin-thoma.de (Martin Thoma) — Sun, 19 May 2013 15:23:11 +0200

When the circle has exactly the same number of blue / green dots in it, it will be green.

When you move the mouse over the box, everything will be calculated and drawn again. This leads to flickering with k-means, as k-means includes a random choice of cluster centers.

Changelog

Version	Change
2.2	Cluster centers have the same color as the clustered points; when one cluster has no points (and there are at least as many points as clusters) everything gets recalculated
2.1	users can now specify an arbitrary number of classes; ctrl-key change of class was removed; added hints to configuration options
2.0	k-means implemented
1.0	k-nearest neighbor implemented

Code is on GitHub. You may use it for free, but you should add a link to this article.

The Collatz sequence

info@martin-thoma.de (Martin Thoma) — Thu, 16 May 2013 23:58:45 +0200

The goal of this post is to show you some tools that allow you to visualize data. And I also want to analyze some basic characteristics of the Collatz sequence.

The Collatz sequences $(c^n_i)$ of a number $n \in \mathbb{N}_{> 0}$ is defined like this:

$f:\mathbb{N}_{>0} \rightarrow \mathbb{N}_{> 0}\;\;\;\;f(n) := \begin{cases} \frac{n}{2} & \text{if } n \text{ is even}\\ 3 \cdot n + 1 & \text{if } n \text{ is odd} \end{cases}$

So the sequence $(c^n_{i})$ is defined as:

$c^n_{i} := \begin{cases} n & \text{if } i = 0\\ f(c^n_i) & \text{otherwise} \end{cases}$

You can define a directed graph $G=(V, E)$ like this:

$V = \mathbb{N}_{>0}, \;\;\; E = \{(n, f(n)) | n \in V\}$

I will call this the Collatz graph.

Collatz conjecture

`$\forall_{n \in \mathbb{N}_{>0}} \exists_{i \in \mathbb{N}_{>0}}: c^n_i = 1$`

The Collatz conjecture was not (dis)proved by now. This is astonishing, as it was proposed in 1937 and I think it is very easy to understand.

We also don’t know if the Collatz graph is connected. When it is not connected, it could be that one sequence $(c^n_i)$ goes to infinity or that there is another circle ( $4,2,1,4$ is a circle in the Collatz graph).

Small `$n$`

When you go through all possible Collatz sequences with $n \in 1, \dots, 15$ , this is what you get:

A graph for all Collatz sequences `$(c^n_i)$` with `$n\leq15$`

This image was created with the following Python script:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

# Based on: http://en.wikipedia.org/wiki/File:Collatz-graph-all-30-no27.svg

def f(n):
    if n % 2 == 0:
        return n / 2
    else:
        return 3*n + 1

def writeDotfile(filename, limit, explored):
    dotfile = file(filename, 'w')

    dotfile.write('digraph {\n')
    dotfile.write('node[style=filled,color=".7 .3 1.0"];\n')
    dotfile.write('1\n')
    dotfile.write('node[style=filled,color=".95 .1 1"];\n')
    #dotfile.write('size="15,8";\n')

    for n in range(2, limit):
        while n not in explored:
            dotfile.write(str(n) + ' -> ')
            explored.add(n)
            n = f(n)
        dotfile.write(str(n) + ';\n')
    dotfile.write('}\n')

def createPng(dotfile, base, program):
    import os
    command = program + " -Tsvg " + dotfile + " -o " + base + ".svg"
    print("Execute command: %s" % command)
    os.system(command)

    command = "inkscape "+base+".svg"+" -w 512 --export-png="+base+".png"
    print("Execute command: %s" % command)
    os.system(command)

if __name__ == "__main__":
    import argparse
 
    parser = argparse.ArgumentParser(
        description="Graph for small Collatz sequences"
    )
    parser.add_argument("-f", "--file", dest="filename",
                        default="collatz-graph.gv",
                        help="write dot-FILE", metavar="FILE")
    parser.add_argument("-p", "--program", dest="program",
                  help="dot, neato, twopi, circo, fdp, sfdp, osage", 
                  metavar="PROGRAM", default="dot")
    parser.add_argument("-n", 
                      dest="limit", default=20, type=int, 
                      help="limit")
    args = parser.parse_args()
    
    writeDotfile(args.filename, args.limit, set([1]))
    import os
    createPng(args.filename, os.path.splitext(args.filename)[0], args.program)

called like this:

python small-numbers.py -n 15 -p fdp

`$n=27$`

$n=27$ is an enourmously long sequence:

Collatz sequence `$c^{27}_i$`

It was created with pgfplots:

\documentclass[varwidth=true, border=2pt]{standalone}
\usepackage[margin=2.5cm]{geometry} %layout

\usepackage{pgfplots}

\begin{document}
\begin{tikzpicture}
    \begin{axis}[
            axis x line=middle,
            axis y line=middle,
            enlarge y limits=true,
            scaled y ticks = false,
            width=15cm, height=8cm, % size of the image
            grid = major,
            grid style={dashed, gray!30},
            ylabel=`$c^{27}_i$`,
            xlabel=`$i$`,
            legend style={at={(0.1,-0.1)}, anchor=north}
         ]
          \addplot[sharp plot, mark=x, blue] table [x=steps, y=n, col sep=comma] {../collatz27.csv};
    \end{axis}
\end{tikzpicture}
\end{document}

How long are Collatz sequences?

I’ve been interested in the question how long Collatz sequences are. Of course, they will be longer when $n$ is bigger. But how does the choice of $n$ influence the number of steps it takes until you reach $c^n_i = 1$ ?

I’ve tested all Collatz sequences with $n \leq 10,000,000$ . This is the result:

Collatz sequence steps

For every hexagon, you check how many datapoints $(n,steps)$ you have there. This leads to the count. As you can see, step numbers from 50-120 are very common, the rest is very uncommon. The number of steps increases very slow.

The data was created as a 116.9 MB csv file with this C++ code:

#include <iostream>
#include <string>
#include <map>
#include <vector>
#include <climits> // get maximum value of unsigned long long
#include <cstdlib> // exit

#define SURPRESS_OUTPUT true
#define SHOW_DICT_CREATION false
 
using namespace std;

struct element {
    /** What is the next collatz number? */
    unsigned long long next;
    
    /** How many steps does it take until you reach 1? */
    unsigned long long steps;
};

map<unsigned long long, struct element> collatz;

unsigned long long CRITICAL_VALUE = (ULLONG_MAX-1) / 3;

unsigned long long maxAddFromOneEntry = 0;
unsigned long long maxEntry = 0;
unsigned long long maxStepsToOne = 0;
unsigned long long saveULong = 0;

/** n >= 1 */
unsigned long long nextCollatz(unsigned long long n) {
    if (n%2 == 0) {
        return n/2;
    } else {
        if (n >= CRITICAL_VALUE) {
            cerr << "Critical value is: " << CRITICAL_VALUE << endl;
            cerr << "n is: " << n << endl;
            cerr << "saveULong is: " << saveULong << endl;
            exit(1);
        }
        return 3*n+1;
    }
}

void insertCollatz(unsigned long long i){
    if (collatz.find(i) == collatz.end()) {
        if (SHOW_DICT_CREATION && !SURPRESS_OUTPUT) {
            cout << i << " is not in collatz:" << endl;
        }

        // i is not in collatz
        vector<unsigned long long> steps;
        unsigned long long current = i;
        unsigned long long next = nextCollatz(current);
        while(collatz.find(current) == collatz.end()) {
            steps.push_back(current);
            current = next;
            next = nextCollatz(current);
        }

        if (steps.size() > maxAddFromOneEntry) {
            maxAddFromOneEntry = steps.size();
        }

        vector<unsigned long long>::reverse_iterator it;
        for(it=steps.rbegin(); it != steps.rend(); it++){
            struct element el;
            el.next = current;
            el.steps = collatz[current].steps + 1;
            collatz[*it] = el;

            if (el.steps > maxStepsToOne) {
                maxStepsToOne = el.steps;
            }

            if (*it > maxEntry) {
                maxEntry = *it;
            }

            current = *it;

            if (SHOW_DICT_CREATION && !SURPRESS_OUTPUT) {
                cout << "\tinserted " << *it << "->" << el.next << endl;
            }
        }

        return;
    } else if (SHOW_DICT_CREATION && !SURPRESS_OUTPUT) {
        cout << i << " was already in collatz." << endl;
    }
}

void printCollatz() {
    for(map<unsigned long long, struct element>::iterator it=collatz.begin(); 
        it!=collatz.end(); ++it) {
        unsigned long long next = (*it).first;
        while(next != 1) {
            cout << next << "->";
            next = collatz[next].next;
        }
        cout << 1 << endl;
    }
}

void printSteps(unsigned long long max) {
    cout << "n,steps" << endl;
    for(unsigned long long i=1;i<=max;i++) {
        cout << i << "," << collatz[i].steps << endl;
    }
}

int main(int argc, char* argv[]) {
    struct element e;
    e.next = 4;
    e.steps = 0;
    collatz[1] = e;

    unsigned long long maxCollatz = (unsigned long long) atoi(argv[1]);
 
    for (unsigned long long i = 2; i <= maxCollatz; i++) {
        insertCollatz(i);
        saveULong = i;
        if (i % 1000000 == 0) {
            cerr << i << endl;
        }
    }

    cerr << "maxAddFromOneEntry: " << maxAddFromOneEntry << endl;
    cerr << "maxStepsToOne: " << maxStepsToOne << endl;
    cerr << "maxEntry: " << maxEntry << endl;
    cerr << "entries: " << collatz.size() << endl;
    
    //printCollatz();
    printSteps(maxCollatz);

    return 0;
}

Then I’ve processed it with R:

R -f analyze.R

analyze.R:

library(ggplot2)
memory.limit(4000)

mydata = read.csv("/home/moose/Downloads/algorithms/collatz/steps.csv")

# Prepare data
p<-ggplot(mydata, aes ( x=n,y=steps ))

p<-p + geom_hex(bins=30)
p<-p + opts(panel.background = theme_rect(fill='white', colour='white'))

# This will save the result in a pdf file called Rplots.pdf
p

And finally, I’ve converted it to png:

inkscape Rplots.pdf -w 512 --export-png=collatz-sequence-steps.png

I’ve explained this a bit more detailed on StackExchange.

Maximum in sequence

In the following plot you can see $n \in 1, \dots, 10,000,000$ on the $x$ -axis and the maximum $y = \max(\{a^n_i | i \in \mathbb{N}_{> 0}\})$ :

Hexagonal binpacking plot for maximum in sequence

library(ggplot2)

mydata = read.csv("../collatz-maxNumber.csv")

# Prepare data
p<-ggplot(mydata, aes(x=n, y=maximum)) + scale_y_log10()

p<-p + geom_hex(bins=50)
p<-p + opts(panel.background = theme_rect(fill='white', colour='white'))

# This will save the result in a pdf file called Rplots.pdf
p

Execution times

Generating all Collatz sequences up to 10,000,000 items took about 50 seconds. But R needed about 10 minutes to generate images from that.

Inkscape didn’t like the heavy plot:

moose@pc07$ inkscape Rplots.pdf -w 512 --export-png=maxInSequence.png

(inkscape:26733): GLib-ERROR **: /build/buildd/glib2.0-2.34.1/./glib/gmem.c:165: failed to allocate 3440640 bytes
^CTrace/breakpoint trap (core dumped)

Maximum in sequence and steps

Maximum value and number of steps for n up to 10,000

All sources of this article are on GitHub
Dot guide, Node shapes
R on UbuntuUsers (German)
Project Euler 14

Maps in C++

info@martin-thoma.de (Martin Thoma) — Thu, 16 May 2013 08:52:54 +0200

Maps are one of the most useful datastructures in C++ and there is no excuse for not knowing it.

Here is a basic example that shows how you can use it:

#include <iostream> // cout
#include <string>

#include <map>

using namespace std;

int main() {
    map<string, string> phonebook;

    // Put some stuff in it
    phonebook["Martin"] = "(0123) 45 678";
    phonebook["Alice"] = "+(13) 37 0000";
    phonebook["Bob"]   = "+(13) 37 0000";
    phonebook["Charlie"] = "Alice";

    // Look stuff up
    cout << "The phone number of Alice is " 
         << phonebook["Alice"] << endl;

    cout << "Number of phone book entries: "
         << phonebook.size() << endl;

    // Print everything
    cout << "Iterate over all phonebook entries: " << endl;
    for(map<string,string>::iterator it=phonebook.begin(); 
        it!=phonebook.end(); ++it) {
        cout << "\t" << (*it).first << ": " << (*it).second << endl;
    }

    // Check if entry exists:
    string person = "Bob";
    cout << "Does " << person << " have a phone number ?" << endl;
    map<string,string>::iterator it = phonebook.find(person);
    if(it != phonebook.end()) {
        //element found:
        cout << "\t" << "Yes! His number is: " << it->second << endl;
    } else {
        cout << "\t" << "No." << endl;
    }
}

Google Code Jam – Round 1C 2013

info@martin-thoma.de (Martin Thoma) — Sun, 12 May 2013 15:01:15 +0200

Problem A (Consonants):
- Small Set: 4305/4834 users (89%)
- Large Set: 1551/3778 users (41%)
Problem B (Pogo):
- Small Set: 2537/3129 users (81%)
- Large Set: 121/638 users (19%)
Problem C (The Great Wall):
- Small Set: 934/1260 users (74%)
- Large Set: 74/330 users (22%)

go-hero.net

Consonants

nip

#!/usr/bin/env python
# -*- coding: utf-8 -*-

def solve(s, n):
    vowels = {'a', 'e', 'i', 'o', 'u'}
    nvalue = 0
    count = 0 # how many consecutive consonants
    pos = -1 # position of the last substring of n consonants
    for i, c in enumerate(s):
        if c in vowels:
            count = 0
        else:
            count += 1
        if count >= n:
            pos = i + 2 - n
        if pos >= 0:
            nvalue += pos
    return nvalue
 
if __name__ == "__main__":
    testcases = input()
      
    for caseNr in xrange(1, testcases+1):
        name, n = raw_input().split(" ")
        print("Case #%i: %s" % (caseNr, solve(name, int(n))))

Pogo

#!/usr/bin/env python
# -*- coding: utf-8 -*-

def calculateSteps(x, y):
    s = 0
    dist = abs(x) + abs(y)
    while (s**2 + s)/2 < dist or ((s**2 + s)/2)%2 != dist%2:
        s += 1
    return s
 
def solve(x,y):
    """ starting at (0|0) and going i steps, 
        how can you reach (x|y)? """   
    s = calculateSteps(x, y)
 
    solution = ""
    for i in range(s, 1-1,-1):
        if abs(x) > abs(y):
            if x > 0:
                solution += "E"
                x -= i
            else:
                solution += "W"
                x += i
        else:
            if y > 0:
                solution += "N"
                y -= i
            else:
                solution += "S"
                y += i
    return solution[::-1]

if __name__ == "__main__":
    testcases = input()
 
    for caseNr in xrange(1, testcases+1):
        x,y = raw_input().split(" ")
        x,y = int(x),int(y)
        print("Case #%i: %s" % (caseNr, solve(x,y)))

The Great Wall

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from collections import defaultdict

def prepareTribes(tribes):
    tribeStack = []
    for tribe in tribes:
        for attackNumber in range(0, tribe["ni"]):
            tribeStack.append({
                "day" :tribe["di"]+attackNumber*tribe["delta_di"],
                "west":2*(tribe["wi"]+attackNumber*tribe["delta_pi"]),
                "east":2*(tribe["ei"]+attackNumber*tribe["delta_pi"]),
                "height":tribe["si"]+attackNumber*tribe["delta_si"]
            })
    return sorted(tribeStack, key=lambda tribe: tribe["day"])

def runAttack(wall, tribe):
    increase = []
    for i in xrange(tribe["west"], tribe["east"] + 1):
        if wall[i] < tribe["height"]: # wall-ee
            increase.append({"wallPos" : i, "height" : tribe["height"]})

    return increase

def solve(tribes):
    wall = defaultdict(int)
    tribeStack = prepareTribes(tribes)
    #for tribe in tribeStack:
    #    print tribe["day"], "[" + str(tribe["west"]) + "," + str(tribe["east"])+"]", tribe["height"]
    successes = 0
    increase = []
    for i, tribe in enumerate(tribeStack):
        increaseTmp = runAttack(wall, tribe)
        #print wall
        #print tribe
        if len(increaseTmp) > 0:
            successes += 1

        increase += increaseTmp

        if i+1==len(tribeStack) or tribeStack[i+1]["day"] > tribe["day"]:
            for el in increase:
                if wall[el["wallPos"]] < el["height"]:
                    wall[el["wallPos"]] = el["height"]
    return successes

if __name__ == "__main__":
    testcases = input()
      
    for caseNr in xrange(1, testcases+1):
        N = input() # Number of tribes attacking the wall
        tribes = []
        for tribe in range(N):
            di, ni, wi, ei, si, delta_di, delta_pi, delta_si = raw_input().split(" ")
            tribes.append({"di":int(di), # the day of the tribe's first attack
            "ni": int(ni), # the number of attacks from this tribe
            "wi": int(wi), # the westmost 
            "ei": int(ei), # and eastmost points respectively of the Wall attacked on the first attack
            "si": int(si), # the strength of the first attack
            "delta_di": int(delta_di), # the number of days between subsequent attacks by this tribe
            "delta_pi": int(delta_pi), # the distance this tribe travels to the east between subsequent attacks (if this is negative, the tribe travels to the west)
            "delta_si": int(delta_si) # the change in strength between subsequent attacks
            })
        print("Case #%i: %s" % (caseNr, solve(tribes)))

Java solution

How do hash functions work?

info@martin-thoma.de (Martin Thoma) — Sat, 11 May 2013 20:07:07 +0200

Everybody who has written a noticeable amount of Java code should know the method hashCode(). But most beginners have difficulties to understand the significance of this little method. The following article gives you one small example with some impressions how much hash functions influence execution time.

Connect four

Connect Four [...] is a two-player game in which the players first choose a color and then take turns dropping colored discs from the top into a seven-column, six-row vertically-suspended grid. The pieces fall straight down, occupying the next available space within the column. The object of the game is to connect four of one's own discs of the same color next to each other vertically, horizontally, or diagonally before your opponent.

Source: Wikipedia

It looks like this:

Connect Four
Source: commons.wikimedia.org

The task

Imagine you would like to find a good strategy where to drop your disk. A simple brute-force method is to create a so called game tree. This means you go through each possibility at each situation that could occur in the game for both players.

This approach has generally two problems:

You have to know how to go through each situation. For connect four it is easy. Both players place their disks in turns and in every turn the current player has at most 7 possibilities. But it is impossible for games like Calvinball or Mao.
The game tree might be HUGE. In this case, you can have `$4,531,985,219,092 \approx 4.5 \cdot 10^{12}$` game situations (source). Even if you would need only one bit for each situation, it would require 566.5 GB!

Anyway, lets say we want to store many unique game situations. Unique means, even if you have hundreds of possible paths to get to a given game situations, you will store this game situation only once.

Implementation

First of all, I would like to mention that you can skip the source code. I’ve only included it to make it easier to understand what I’m talking about.

Lets say our game situation looks like this:

struct gamesituation {
    /** How does the board currently look like? */
    char board[BOARD_WIDTH][BOARD_HEIGHT];

    /**
     * What are the next game situations that I can reach from this 
     * board? 
     * The next[i] means that the player dropped the disc at column i
     */
    int next[7];

    /* I could use a bitfield for this ... but it would make access
     * much more inconvenient. 
     */
    unsigned char isEmpty;  // Is this gamesitatution already filled?
    unsigned char isFinished; // Is this game finished?
    unsigned char stalemate; // Was this game a stalemate?
    unsigned char winRed;   // Did red win?
    unsigned char winBlack; // Did black win?
};

You need a check if one player won:

/*
 * Check if player has won by placing a disc on (x,y). 
 * with direction (xDir, yDir)
 * @return 1 iff RED won, -1 iff BLACK won and 0 if nobody won
 */
signed char hasPlayerWon(char board[BOARD_WIDTH][BOARD_HEIGHT], 
                  int x, int y, char xDir, char yDir) {
    char color = board[x][y];

    int tokensInRow = getTokensInRow(board, color, x, y, xDir, yDir)
              + getTokensInRow(board, color, x, y, -xDir, -yDir) - 1;

    if (tokensInRow >= WINNING_NR) {
        if (color == RED) {
            return 1;
        } else if (color == BLACK) {
            return -1;
        } else {
            perror("this color doesn't / shouldn't exist\n");
            exit(1);
        }
    }

    return 0;
}

/* 
 * A new disc has been dropped. Check if this disc means that 
 * somebody won.
 * @return 1 iff RED won, -1 iff BLACK won, otherwise NOT_FINISHED
 */
int isBoardFinished(char board[BOARD_WIDTH][BOARD_HEIGHT], 
                    int x, int y) {
    signed char status;

    // check left-right
    status = hasPlayerWon(board, x, y, 1, 0);

    if (status != 0) {
        return status;
    }

    // top-down
    status = hasPlayerWon(board, x, y, 0, 1);

    if (status != 0) {
        return status;
    }

    // down-left to top-right
    status = hasPlayerWon(board, x, y, 1, 1);

    if (status != 0) {
        return status;
    }

    // top-left to down-right
    status = hasPlayerWon(board, x, y, -1, 1);

    if (status != 0) {
        return status;
    }

    return NOT_FINISHED;
}

If you need an explanation for this, you should read this article.

And you need a function that can mirror boards (to get rid of identical, but mirrored situations) and one that can compare boards:

char isSameBoard(char a[BOARD_WIDTH][BOARD_HEIGHT], 
                 char b[BOARD_WIDTH][BOARD_HEIGHT]) {
    for (int x = 0; x < BOARD_WIDTH; x++) {
        for (int y = 0; y < BOARD_HEIGHT; y++) {
            if (a[x][y] != b[x][y]) {
                return FALSE;
            }
        }
    }

    return TRUE;
}

void mirrorBoard(char board[BOARD_WIDTH][BOARD_HEIGHT],
                 char newBoard[BOARD_WIDTH][BOARD_HEIGHT]) {
    for (int x = 0; x < BOARD_WIDTH; x++) {
        for (int y = 0; y < BOARD_HEIGHT; y++) {
            newBoard[BOARD_WIDTH - x - 1][y] = board[x][y];
        }
    }
}

You need a function that makes all possible moves for the players:

/*
 * Make all possible turns that the player can make in this
 * game situation.
 */
void makeTurns(char board[BOARD_WIDTH][BOARD_HEIGHT], 
   char currentPlayer, unsigned int lastId, int recursion) {
    unsigned int insertID;
    int outcome;

    for (int column = 0; column < BOARD_WIDTH; column++) {
        // add to column
        int height = BOARD_HEIGHT - 1;

        // the disc falls down
        while (height >= 0 && board[column][height] == EMPTY) {
            height--;
        }
        height++;

        // this colum is full
        if (height == 6) {
            continue;
        }

        // place disc
        board[column][height] = currentPlayer;

        if (didBoardAlreadyOccur(board)) {
            // I've already got to this situation
            insertID = getBoardIndex(board);
            savePreviousID(insertID, lastId, column);
        } else {
            char mirrored[BOARD_WIDTH][BOARD_HEIGHT];
            mirrorBoard(board, mirrored);

            if (didBoardAlreadyOccur(mirrored)) {
                // I've already got this situation, but mirrored
                // so take care of symmetry at this point
                mirroredCounter++;
                insertID = getBoardIndex(mirrored);
                savePreviousID(insertID, lastId, column);
            } else {
                registeredSituations++;

                if (registeredSituations == MAXIMUM_SITUATIONS) {
                    giveCurrentInformation();
                    exit(MAXIMUM_SITUATIONS_REACHED_EXIT_STATUS);
                }

                if (REGISTERED_MOD > 0 &&
                    registeredSituations % REGISTERED_MOD == 0) {
                    giveCurrentInformation();
                }

                outcome = isBoardFinished(board, column, height);

                if (ABS(outcome) <= 1) {
                    // the game is finished
                    insertID = getNewIndex(board);
                    storeToDatabase(insertID, board, TRUE, outcome);
                    savePreviousID(insertID, lastId, column);
                } else {
                    // Switch players
                    if (currentPlayer == RED) {
                        currentPlayer = BLACK;
                    } else {
                        currentPlayer = RED;
                    }

                    insertID = getNewIndex(board);
                    setBoard(insertID, board);
                    savePreviousID(insertID, lastId, column);
                    char copy[BOARD_WIDTH][BOARD_HEIGHT];

                    for (int x = 0; x < BOARD_WIDTH; x++) {
                        for (int y = 0; y < BOARD_HEIGHT; y++) {
                            copy[x][y] = board[x][y];
                        }
                    }

                    makeTurns(copy, currentPlayer, insertID, 
                              recursion + 1);
                }
            }
        }
    }
}

How is this realated to hash functions?

You might have noticed a few functions that I didn’t explain by now:

didBoardAlreadyOccur(board): Checks if a given board is stored in database.
getBoardIndex(board): This is a function that takes a board and gives a non-negative integer which is characteristic for the given board.
savePreviousID(insertID, lastId, column): store insertID as a possible next situation for lastId in database
setBoard(insertID, board): Insert board into database at position insertID

How would you implement didBoardAlreadyOccur(board)? This function (or insertID) will be the slowest part of the code and will be called VERY often. So it needs to be as fast as possible.

A hash function

Most of the time you can create hash functions by mapping values to integers. In my case, I mapped the board - which is a two-dimensional char array - to one integer by thinking of it as a very long number. I think of a red disc as the digit 1, a black disc as the digit 2 and an empty field as 0:

unsigned int charToInt(char x) {
    if (x == RED) {
        return 1;
    } else if (x == BLACK) {
        return 2;
    } else {
        return 0;
    }
}

When you want to get the board number, you can get it like this:

Empty=0, red=1, yellow=2
The board number is 00000000000000000210000211210112212

For most game situations, this number will be much too big to store it in an integer. Also, we would like to get an index for our array so that we know where to store this board. The simplest solution to this problem is to calculate NUMBER % ARRAY_SIZE:

unsigned int getFirstIndex(char board[BOARD_WIDTH][BOARD_HEIGHT]) {
    unsigned int index = 0;

    for (int x = 0; x < BOARD_WIDTH; x++) {
        for (int y = 0; y < BOARD_HEIGHT; y++) {
            index += charToInt(board[x][y]) * 
                     myPow(3, ((x + y * BOARD_WIDTH) % HASH_MODULO));
        }
    }

    index = index % MAXIMUM_SITUATIONS;
    return index;
}

The function getFirstIndex maps an char Array with BOARD_WIDTH * BOARD_HEIGHT = 7 * 6 = 42 elements to an integer interval [0, MAXIMUM_SITUATIONS] = [0, 20000000]. Although I only use three values for the char array, that is $3^{42} = 109418989131512359209 \approx 1.09 \cdot 10^{20}$ . There are many game situation numbers that can never occur (e.g. two more red than black dists), but we still map a significantly larger space to [0,20000000]. You can’t change that. You can probably find (much) better mappings, but as we know that there are $4.5 \cdot 10^{12}$ game situations, you will always have the problem that your codomain is much smaller than the domain of your hash function. That’s a fundamental problem of hash functions.

This means, you will have two board situations that map to the same hash number. This is called a “hash collision”. When you use the hash number directly as an index for your board, you will have to deal with hash collisions. Some solutions are:

Ignoring the problem: That's boring and not always possible (but simple).
Linear probing
Quadratic probing

Linear probing

The idea of linear probing is very simple:

Inserting a new item:

You look at the index `$i$` that your hash function gave you.
If this index is already full, you look at `$i+1$`
When you took a look at all slots of your array, you can't insert the new item.

Searching for an already inserted item:

You look at the index `$i$` that your hash function gave you.
If `$i$` is empty, you're ready. The item was not inserted.
If `$i$` is not the item you've searched for, you have to look at `$i+1$`.
Keep looking at the next item until you find your searched item, you've looked at all items or you find an empty slot.

Deleting is complicated. You have to look at all items after the deleted one, remove them from your array and insert them again. That’s not good.

The problem of linear probing is clustering. When you have some hash values that are close together, you might get hash collisions faster. When you’ve got your first collisions, you resolve them by inserting the value close to the value where you originally wanted to save it. So you get one big cluster quite fast. When you want to insert an element in the cluster, you first have to search the end of the cluster. That’s bad for performance.

An advantage of linear probing compared to quadratic probing is that you might get a better performance due to cache effects.

Quadratic probing

The idea of quadratic probing is the same as for linear probing, but you try to fix the clustering-problem by using a clever way to search for a free spot:

$h_i(x) = \left(h(x) + (-1)^{i+1} \cdot \left\lceil\frac{i}{2}\right\rceil^2\right) \bmod~m$

where $h$ is your hash function and $i$ is your i-th try to find a free spot while you have $m$ spots in total.

This one also suffers from clustering, but it’s not as bad as with linear clustering.

Double hashing

This solution could be the best one, but also the hardest one to implement correctly. You could find a free spot by using a second hash function $h'$ like this:

$h_i(x) = (h(x)+h'(x)\cdot i) ~ \bmod ~ m$

BUT you have to make sure that $Pr[h(x)=h(y) \land h'(x)=h'(y)] = \frac{1}{m^2}$

Performance

You can use linear probing, quadratic probing and double hashing in my example and measure how many game situations get stored. The more game situations you can store in the same amount of time, the better:

Linear probing, quadratic probing and double hashing for connect four

You can see that linear probing performs much worse than quadratic probing and double hashing. When you compare quadratic probing with double hashing, there seems not to be a big difference. But note that my second hash function is almost the same as the first one. You could probably choose a better second hash function and get better results (suggestions are welcome).

Why are hash functions important?

Hash functions help you to map a big amount of data to a small space. They are important, because they are a relevant part of many datastructures. The better they are, the faster will operations on those datastructures work. Better means: Faster to compute or less collisions.

Some datastructures like this are:

HashMap and HashTable (→ difference)
HashSet

Final notes

Another resolution for hash collisions is creating a linked list. This means you will not suffer from clustering and you can insert in $\mathcal{O}(1)$ . But searching for an element is still in $\mathcal{O}(n)$ , where $n$ is the number of elements that were already inserted.

Resources

You can find the code at GitHub.

Google Code Jam – Round 1B 2013

info@martin-thoma.de (Martin Thoma) — Sun, 05 May 2013 16:15:32 +0200

Problem A (Osmos):
- Small Set: 4668/7250 users (64%)
- Large Set: 3537/4578 users (77%)
Problem B (Falling Diamonds):
- Small Set: 952/1882 users (51%)
- Large Set: 525/724 users (73%)
Problem C (Garbled Email):
- Small Set: 444/896 users (50%)
- Large Set: 255/345 users (74%)

go-hero.net

Osmos

#!/usr/bin/env python
# -*- coding: utf-8 -*-

def howBigDoIget(A, motes):
    while len(motes) > 0 and A > min(motes):
        A += min(motes)
        motes.remove(min(motes))
    return A

def stepsNeededForNext(A, motes):
    m = min(motes)
    steps = 0
    if m >= 1 and A == 1:
        return 10**12
    while A <= m:
        A += (A-1)
        steps += 1
    return steps

def solve(A, motes):
    steps = 0
    A = howBigDoIget(A, motes)
    while len(motes) > 0 and A <= max(motes):
        if (stepsNeededForNext(A, motes) >= len(motes)):
            steps += len(motes)
            return steps
        else:
            A += (A-1)
        A = howBigDoIget(A, motes)
        steps += 1
    return steps
 
if __name__ == "__main__":
    testcases = input()
    for caseNr in xrange(1, testcases+1):
        A, N = map(int,raw_input().split(" "))
        motes = sorted(map(int,raw_input().split(" ")))
        copyed = motes[:]
        solution = solve(A, motes)
        if solution > N:
            solution = N
        print("Case #%i: %s" % (caseNr, solution))

Falling Diamonds

First of all, diamonds only fall at `$x=0$`!
If your target coordinates are `$(x,y)$`, you have the same output as for `$(-x,y)$`, as everything is symmetric.
You have to get a basis for your diamonds pyramid. I've colored the basis in yellow in the images below.
When your target is above the ground, you can let the diamond slide down to calculate the size of the basis.

You have to get those yellow diamonds first, before you can get the orange one.

Let Diamonds slide down

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import gmpy

""" Calculate the binomial coefficient """
def binomial(n, k):
    return gmpy.comb(n,k)

""" 
    @param N: Number of diamonds
    @param x,y: Target coordinate
    @return: possiblity, that a diamond will be at coordinate (x,y) 
"""
def solve(N, x, y):
    if x == 0:
        n = y+1
        if N >= (n*n+n)/2:
            return 1.0
        else:
            return 0.0

    # From this point, x != 0 is True

    xTmp = x + y # let target slide down

    n = xTmp-1
    baseDiamands = (n**2+n)/2

    # are there enough diamonds left after you've build the basis?
    rest = N - baseDiamands
    if rest <= 0:
        return 0.0

    # are there enough diamonds left so that you can guarantee that 
    # you will  fill up the next bigger pyramid at least to the 
    # target position?
    biggerBaseDiamonds = baseDiamands+n+2+y
    if N >= biggerBaseDiamonds:
        return 1.0

    # some math:
    # bernoulli
    prob = 0.0
    hitsNeeded = y+1

    for k in range(hitsNeeded, rest+1):
        prob += binomial(rest,k)
    
    return prob/2**rest
 
if __name__ == "__main__":
    testcases = input()
      
    for caseNr in xrange(1, testcases+1):
        N, x, y = map(int,raw_input().split(" "))
        print("Case #%i: %.9Lf" % (caseNr, solve(N, abs(x), y)))

Sicherheit-Klausur

info@martin-thoma.de (Martin Thoma) — Mon, 29 Apr 2013 10:19:05 +0200

Dieser Artikel beschäftigt sich mit der Vorlesung „Sicherheit“ am KIT. Er dient als Prüfungsvorbereitung. Ich habe die Vorlesungen bei Herrn Jun.-Prof. Hofheinz im Sommersemester 2013 gehört.

An diesem Artikel wird natürlich noch gearbeitet.

Behandelter Stoff

25.04.2013

Caesar, Vigenere, One-Time-Pad

VL 01

22.04.2013

Symmetrische Verschlüsselungen; Stromchiffren; Blockchiffren (DES, AES); Betriebsmodi:

Modus	Verschlüsselung Entschlüsselung	Parallelisierbarkeit
Modus	Verschlüsselung Entschlüsselung	verschl.	entschl.
ECB	`$c_i = ENC(K, m_i)$` `$m_i = DEC(K, m_i)$`	Ja	Ja
CBC	`$c_i = ENC(K, m_i \oplus c_{i-1})$` `$m_i = DEC(K, c_i) \oplus c_{i-1}$`	Nein	Ja
CFB	`$c_i = m_i \oplus ENC(K, c_{i-1})$` `$m_i = c_i \oplus ENC(K, c_{i-1})$`	Nein	Ja
OFB	`$c_i = m_i \oplus ENC(K, IV)^i$` `$m_i = c_i \oplus ENC(K, IV)^i$`	Nein	Nein

DES-Feistelstruktur

VL 02

25.04.2013

Lineare Kryptoanalyse, Differentielle Kryptoanalyse, Semantische Sicherheit, IND-CPA, Feistel-Schema

VL 03

29.04.2013

Hashfunktionen, Kollisionsresistenz `$\Rightarrow$` Einwegeigenschaft, Merkle-Damgard-Konstruktion und Length-Extension-Problematik, Birthday Attack, Meet-in-the-Middle-Angriff

VL 04

06.05.2013

Public-Key-Verschlüsselung (Idee, RSA); Meet-in-the-Middle-Angriff für Hashfunktionen

VL 05

13.05.2013

Rest Public-Key-Verschlüsselung (ElGamal, Vergleich RSA/ElGamal), Symmetrische Authentifikation von Nachrichten (Sicherheitsmodell, Hash-then-Sign, PRFs, HMAC), MAC

VL 06

27.05.2013

Asymmetrische Authentifikation von Nachrichten (RSA-PSS, ElGamal, DSA)

VL 07

03.06.2013

Schlüsselaustausch (Kerberos, TLS, Angriffe)

VL 08

10.06.2013

Schlüsselaustausch (TLS-Angriffe, weitere Verfahren), Identifikationsprotokolle

VL 09

13.06.2013

Identifikationsprotokolle (Sicherheitsanalyse), Zero-Knowledge-Protokolle, Commitment → Hiding

VL 10

17.06.2013

Zero-Knowledge-Protokolle, Nutzerauthentifikation

VL 11

24.06.2013

Nutzerauthentifikation (Wörterbuchangriffe, interaktive Authentifikation, positionsbasierte Kryptographie); Rainbowtables; LM-Hashes

VL 12

27.06.2013

Nutzerauthentifikation (positionsbasierte Kryptographie), Zugriffskontrolle (Bell-LaPadula)

VL 13

01.07.2013

Zugriffskontrolle (Bell-LaPadula, Chinese Wall), Analyse größerer Systeme

VL 14

08.07.2013

Analyse größerer Systeme, häufige Sicherheitsprobleme

VL 15

11.07.2013

Häufige Sicherheitsprobleme

VL 16

Falls hier was fehlt, könnt ihr mich gerne in den Kommentaren oder per Mail (info@martin-thoma.de) darauf aufmerksam machen. Ich bin ja mal gespannt, ob ich das bis zum Ende aktuell halte.

Definitionen und Sätze

Eine über `$k$` parametrisierte Funktion `$H$` ist kollisionsresistent, wenn jeder PPT-Algorithmus nur mit höchstens vernachlässigbarer Wahrscheinlichkeit eine Kollision findet. Für jeden PPT-Algorithmus `$\mathcal{A}$` ist `$Adv^{cr}_{H,\mathcal{A}}(k) := Pr[(X,X') \leftarrow \mathcal{A}(1^k): X \neq X' \land H_k(X) = H_k(X')]$` vernachlässigbar.

Eine über `$k$` parametrisierte Funktion `$H$` ist eine Einwegfunktion bzgl. der Urbildverteilung `$\mathcal{X}_k$`, wenn jeder PPT-Algorithmus nur mit höchstens vernachlässigbarer Wahrscheinlichkeit ein Urbild eines gegebenen, aus `$\mathcal{X}_k$` gezogenen Bildes findet. Für jeden PPT-Algorithmus `$\mathcal{A}$` ist `$Adv^{cr}_{H,\mathcal{A}}(k) := Pr[X' \leftarrow \mathcal{A}(1^k, H(X)): H(X) = H(X')]$` vernachlässigbar, wobei `$X \leftarrow \mathcal{X}_k$` gewählt wurde.

Jede kollisionsresistente Hashfunktion `$H:\{0,1\}^* \rightarrow \{0,1\}^k$` ist eine Einwegfunktion bzgl. der Gleichverteilung auf `$\{0,1\}^{2k}$`.

Ist `$f$` eine kollisionsresistente Hashfunktion, so ist die Merkle–Damgård-Konstruktion mit `$f$` kollisionsresistent.

Hash-Then-Sign-Paradigma: Sei (Sig, Ver) EUF-CMA-sicher und H eine kollisionsresistente Hashfunktion. Dann ist der durch Sig'(K, M) = Sig(K, H(M)), Ver'(K, M, `$\sigma$`) = Ver(K, H(M), `$\sigma$`) erklärte MAC EUF-CMA-sicher.

Fragen

Wann ist ein Verschlüsselungsschema IND-CPA-sicher?

IND-CPA bedeutet „indistinguishability under chosen-plaintext attacks“. Ein Verschlüsselungsschema ist genau dann IND-CPA-Sicher, wenn kein effizienter Angreifer `$\mathcal{A}$` Chiffrate von selbstgewählten Klartexten unterscheiden kann.

Wann ist eine Signatur EUF-CMA-sicher?

EUF-CMA bedeutet „existentially unforgeable under chosen-message attacks“. Eine Signatur ist genau dann EUF-CMA-Sicher, wenn alle PPT-Angreifer `$\mathcal{A}$` folgendes Spiel nur vernachlässigbar oft gewinnen:

`$\mathcal{A}$` hat Zugriff auf ein `$Sig(K, \cdot)$`-Orakel
`$\mathcal{A}$` gibt als Ausgabe `$(M^*, \sigma^*)$`
`$\mathcal{A}$` gewinnt, wenn `$Ver(K, M^*, \sigma^*) = 1$` und `$M^*$` „frisch“ ist

Was sind Replay-Angriffe?

Bei Replay-Angriffen fängt der Angreifer einen Teil der Kommunikation von Alice und Bob ab. Später schickt er diesen Teil ohne weitere Bearbeitung an einen der Beiden.

Was versteht man unter der Merkle–Damgård-Konstruktion?

Die Merkle–Damgård-Konstruktion ist eine Methode zur Konstruktion von kryptographischen Hash-Funktionen. Sie funktioniert so:

Merkle-Damgard-Konstruktion
Quelle: Wikipedia

Wie funktioniert RSA?

Generiere zwei Primzahlen `$p, q \in \mathbb{P}$`
Berechne `$n := p \cdot q$` und `$\varphi(n) = (p-1) \cdot (q-1)$`
Wähle `$e$`, sodass gilt: `$ggT(e, \varphi(n)) = 1$` und `$1 < e < \varphi(n)$`
Berechne `$d$`, sodass gilt: `$e \cdot d \equiv 1 \mod \varphi(n)$`

Verschlüsselung einer Nachricht `$m$`: `$c = m^{e} \mod n$` Verschlüsselung eines Ciphertextes `$c$`: `$m = c^{d} \mod n$`

Welches Problem birgt ein kleines `$e$` beim RSA-Verfahren?

Folgender Angriff ist für `$e=3$` möglich:

Nachricht `$m$` wird an 3 Benutzer geschickt
Angreifer kennt Chiffrate für `$N_1, N_2, N_3$`.
Chinesischer Restsatz für `$m^3 \mod N_1 N_2 N_3$`.
Wurzelziehen über `$\mathbb{Z}$` liefert `$m$`

Was ist damit gemeint, wenn man sagt „RSA ist Homomorph“?

Homomorphie ist folgende (unerwünschte) Eigenschaft: `$\begin{align} Enc(pk, m_1) \cdot Enc(pk, m_2) &= m_1^e \cdot m_2^e\\ &= (m_1 \cdot m_2)^e \\ &= Enc(pk, m_1 \cdot m_2) \end{align}$` Diese Eigenschaft ist z.B. in folgendem Szenario problematisch: Angenommen bei einer Auktion werden die gebotenen Geldbeträge verschlüsselt. Dann kann ein Angreifer das Chiffrat (gültig) verändern. So kann er den Geldbetrag ohne Probleme verdoppeln.

Was sind HMACs?

Spezielle symmetrische Signaturen, die wie folgt aufgebaut sind: `$Sig(K, M) = H(K \oplus opad, H(K \oplus ipad, M))$`

Wie funktioniert ElGamal?

Wähle eine zyklische Gruppe `$\mathbb{G} = \langle g \rangle$`.
`$pk = (\mathbb{G}, g, g^x)$`, `$sk=(\mathbb{G}, g, x)$`, wobei `$x$` zufällig gewählt wird.

Verschlüsselung einer Nachricht `$m$`: `$Enc(pk, m) = (g^y, g^{xy} \cdot m)$` mit zufälligem `$y$` → Verschlüsselung ist zufällig! Verschlüsselung eines Ciphertextes `$c$`: `$Dec(sk, (Y, Z)) = \frac{Z}{Y^x} = \frac{g^{xy} \cdot m}{(g^{y})^x} = m$`

Wie funktioniert das Kerberos-Schlüsselaustauschprotokoll?

Alice schreibt dem KC, dass Alice und Bob gerne einen Schlüsselaustausch vornehmen wollen.
Das KC schickt Alice ihren Schlüssel `$K_A$`, den sie für die Kommunikation mit Bob verwenden kann. Zusätzlich wird ein Zeitstempel `$T_{KC}$`, eine Gültigkeitsdauer `$L$` sowie ein frischer Schlüssel `$K$` übertragen.
TODO!!!!

Ein gutes YouTube-Video gibts auch.

Warum kann es schlecht für die Sicherheit sein, wenn man komprimiert?

Annahme: Ein Angreifer kann zu dem Geheimtext, der komprimiert wird, etwas vor der Kompression hinzufügen. Wenn der Ciphertext deutlich weniger wächst als er an Text hinzugefügt hat, kann er vermuten, dass er den Text erraten hat. (→ CRIME-Angriff)

Was ist damit gemeint, dass das One-Time-Pad-Chiffre verwundbar ist?

Ein Angreifer kann die Klartextnachricht ändern. Wenn er weiß (oder zumindest ahnt) was der Klartext ist, kann er dafür sorgen, dass ein Klartext gleicher Länge seiner Wahl bei der Entschlüsselung herauskommt.

Nennen Sie Verfahren, die IND-CPA-sicher sind.

Eine Blockciffre im CBC-Modus

Nennen Sie Verfahren, die EUF-CMA-sicher sind.

Sei PRF: `$\{0,1\}^k \times \{0,1\}^k \rightarrow \{0,1\}^k$` eine PRF und `$H:\{0,1\}^* \rightarrow \{0,1\}^k$` eine kollisionsresistente Hashfunktion. Dann ist der durch `$Sig(K, M) = PRF(K, H(M))$` gegebene MAC EUF-CMA-sicher.

Diverses

TLS Handshake

Change Cipher Spec Drop

Ein Angriff auf verschlüsselte Verbindungen:

Change cipher spec drop

Material

Vorlesungswebsite
Mein Anki-Deck
Skript
StackExchange:
- What are the differences between a digital signature, a MAC and a hash?

In der Fachschaft gibt es folgende Altklausuren:

Hauptklausur SS 2012
Nachklausur SS 2011
Hauptklausur SS 2011
Nachklausur SS 2010
Hauptklausur SS 2010

Aufbau der Klausur

Bisher gab es meist 6 Aufgaben mit jeweils 10 Punkten:

Blockchiffren und Betriebsmodi
ElGamal / Diffie-Hellman-Schlüsselaustausch
RSA; RSA-OAEP
Kerberos / Feistel / (H)MAC
Bell-LaPadula / Chinese Wall
Multiple Choice

Übungsbetrieb

Es gibt Übungsblätter auf der Vorlesungswebsite, aber keinen Übungsschein, keine Abgaben und keine Bonuspunkte.

Termine und Klausurablauf

Datum: Freitag, den 26. Juli 2013 von 14:00 bis 15:00 Uhr (Klausur dauerte eine Stunde) Ort: seit 24.07.2013 auf der Vorlesungswebsite (ich bin im H.S.a.F) Punkte: 60 Bestehensgrenze: 20 Übungsschein: Nein Bonuspunkte: Nein

Nicht vergessen

Studentenausweis
Kugelschreiber

Ergebnisse

Sind noch nicht draußen (Stand: 30.07.2013)

Sind nun draußen (Stand: 09.08.2013): Vorläufige Ergebnisse als PDF

Ergebnis der Sicherheitsklausur

Semantische Sicherheit

info@martin-thoma.de (Martin Thoma) — Sun, 28 Apr 2013 15:06:53 +0200

In der Vorlesung vom 25.04.2013 hat Prof. Hofheinz gesagt, dass man semantische Sicherheit praktisch nicht beweisen kann, da man zuerst $\mathcal{P} \neq \mathcal{NP}$ beweisen müsste. Warum das so ist, versuche ich nun zu erläutern.

Einwegfunktionen und $\mathcal{P} \neq \mathcal{NP}$

Sei $f:X \rightarrow Y$ eine Funktion. $f$ heißt eine Einwegfunktion, genau dann wenn für alle $x \in X$ gilt:

$y := f(x)$ kann in Polynomialzeit berechnet werden
Für die Berechnung eines Urbildes $x$ aus $y$ existiert kein randomisierter Algorithmus, der in Polynomialzeit läuft.

Es gilt: Wenn eine Einwegfunktion $f$ existiert, dann gilt $\mathcal{P} \neq \mathcal{NP}$.

Warum?

Nun, angenommen es gibt eine Einwegfunktion $f$. Dann sei die formale Sprache $L_f$ definiert durch:

$L_f := \{(\bar x, y) | \exists x: \bar x \text{ ist Präfix von } x \text{ und } y = f(x)\}$

Es gilt: $L_f \notin \mathcal{P}$, da für ein gegebenes $y$ das zugehörige $x$ in polynomialzeit bestimmt werden könnte (wie will man sonst prüfen, ob $\bar x$ ein Präfix von $x$ ist?)

Falls jemanden diese Begründung nicht ausreicht ist hier noch ein Beweis von Prof. Hofheinz (Danke!)

Beh.: $L_f \notin \mathcal{P}$
Bew.: durch Widerspruch
Annahme.: $L_f \in P$
$\Rightarrow$ Es existiert ein Polyzeit-Algorithmus $\mathcal{A}$ für $L_f$, der bei Eingabe $(\bar x, y)$ entscheidet, ob ein $x$ mit $f(x)=y$ und Präfix $\bar x$ existiert oder nicht. Dann können wir einen Algorithmus $\mathcal{B}$ aus $\mathcal{A}$ bauen, der $f$ invertiert.

Gegeben $y$ verfährt $\mathcal{B}$ wie folgt: $\mathcal{B}$ ruft $\mathcal{A}(0,y)$ auf und erfährt so, ob ein Urbild $x$ von $y$ mit Anfangsbit $0$ existiert. Wenn ja, ruft $\mathcal{B}$ den Algorithmus $\mathcal{A}(00,y)$ auf, wenn nein ruft $\mathcal{B}$ den Algorithmus $\mathcal{A}(10,y)$ auf usw.

So wird ein Urbild $x$ bitweise bestimmt. Ein solches $\mathcal{B}$ findet also effizient Urbilder, im Widerspruch zur Einwegannahme über $f \blacksquare$

Aber: Wenn das $x$ gegeben ist, dann ist es einfach zu zeigen, dass $y= f(x)$ gilt und damit auch, ob $\bar x$ ein Präfix von $x$ ist. Damit ist $L_f \in \mathcal{NP}$.

Damit gilt: $L_f \in \mathcal{NP} \setminus \mathcal{P}$. Wenn aber $\mathcal{NP} \setminus \mathcal{P} \neq \emptyset$, dann gilt insbesondere $\mathcal{P} \neq \mathcal{NP}$.

An dieser Stelle sollte man also einsehen, dass eine Einwegfunktion nach obiger Definition nur existieren kann, wenn $\mathcal{P} \neq \mathcal{NP}$ gilt.

Semantische Sicherheit

Wikipedia gibt folgende Beschreibung von semantischer Sicherheit:

Ein Verschlüsselungsverfahren ist semantisch sicher, wenn jeder Angreifer jede Information, die er aus einem Chiffrat über die Nachricht ableiten kann, bereits dann ableiten kann, wenn er nur die Länge des Chiffrats kennt. Ein Chiffrat verrät also nichts über eine Nachricht als ihre Länge.

Herr Prof. Hofheinz hat folgende informelle Definition von Semantischer Sicherheit in der Vorlesung gegeben:

Ein symmetrisches Verschlüsselungsverfahren ist semantisch sicher, wenn es für jede $M$-Verteilung, jede Funktion $f$ und jeden PPT-Algorithmus $\mathcal{A}$ einen PPT-Algorithmus $\mathcal{B}$ gibt, so dass $Pr \left [\mathcal{A}^{\text{Enc}(K, \cdot)}(\text{Enc}(K, M)) = f(M) \right ] - Pr [\mathcal{B}(\varepsilon) = f(M)]$ vernachlässigbar (als Funktion im Sicherheitsparameter $K$) ist.

Hier ist

$M$ eine Nachricht (Message),
$\text{Enc(K, M)}$ die Verschlüsselung einer konkreten Nachricht $M$ mit dem Schlüssel $K$,
$\varepsilon$ eine triviale Information (ich glaube das ist z.B. die Länge des Ciphertextes) und
$f$ extrahiert beliebige Informationen aus dem Plaintext

Die erste Wahrscheinlichkeit bezeichnet die Möglichkeit, aus dem Ciphertext Informationen der Art $f$ über den Plaintext $M$ zu erhalten. Die zweite Wahrscheinlichkeit bezeichnet die Möglichkeit „aus dem Nichts“ Informationen über eine Nachricht zu erhalten. Damit will man triviale Informationen eliminieren. Insgesamt gibt es also die Wahrscheinlichkeit an, nicht-triviale Informationen aus einer Verschlüsselten Nachricht zu erhalten. Mit effizient ist vermutlich in Polynomialzeit gemeint.

Wenn es nun mehrfach benutzbare semantisch sichere Verfahren gibt, dann kann man dieses Verfahren als Einwegfunktion nutzen. Wenn eine Einwegfunktion existiert, gilt $\mathcal{P} \neq \mathcal{NP}$. Also folgt:

Wenn es nun mehrfach benutzbare semantisch sichere Verfahren existieren, gilt $\mathcal{P} \neq \mathcal{NP}$. Dies ist aber eines der Millennium-Probleme und noch nicht geklärt.

Google Code Jam – Round 1A 2013

info@martin-thoma.de (Martin Thoma) — Sat, 27 Apr 2013 07:15:17 +0200

Problem A (Bullseye):
- Small Set: 5856/6195 users (95%)
- Large Set: 1806/4795 users (38%)
Problem B (Manage your Energy):
- Small Set: 2323/3789 users (61%)
- Large Set: 456/1133 users (40%)
Problem C (Good Luck):
- Small Set: 1366/1774 users (77%)
- Large Set: 31/605 users (5%)

go-hero.net

Bullseye

Small

Large

numpy.roots

#!/usr/bin/env python
# -*- coding: utf-8 -*-
 
from numpy import ceil, roots
 
def check(r, t, x):
    return t-(x+1)*(2*r+2*x+1) >= 0
 
def solveFast(r, t):
    myRoots = roots((2,2*r+3,2*r+1-t))
    for i in xrange(2):
        if myRoots[i] >= 0:
            answer = int(ceil(myRoots[i])) + 1
            while not check(r, t, answer):
                answer -= 1
            return answer + 1
  
if __name__ == "__main__":
    testcases = input()
       
    for caseNr in xrange(1, testcases+1):
        line = raw_input()
        r, t = map(int, line.split(' '))
        print("Case #%i: %s" % (caseNr, solveFast(r, t)))

Good Luck

itertools

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from math import factorial
from itertools import combinations_with_replacement
import pprint
from copy import deepcopy

def mul(integers):
    s = 1
    for p in integers:
        s *= p
    return s

def merge(candidates, block):
    newCandidates = deepcopy(candidates)
    for el in set(block):
        diff = block.count(el) - newCandidates.count(el)
        for i in xrange(diff):
            newCandidates.append(el)
    return newCandidates

def canBeIn(b, candidates, N):
    merged = merge(candidates, b)
    return not (len(merged) > N)

""" 
    N: number of numbers in total that got randomly picked
    M: A_i in [2, M]
"""
def solve(N, M, products, productToBuildungs):
    candidates = []

    # Is there a simple answer?
    for p in products:
        if productToBuildungs[p][0] == 1:
            candidates = merge(candidates, productToBuildungs[p][1][0])
            if len(candidates) == N:
                return candidates

    for p in products:
        pos = filter(lambda b: canBeIn(b, candidates, N), productToBuildungs[p][1])
        if len(pos) == 1:
            candidates = merge(candidates, pos[0])

    while len(candidates) < N:
        candidates.append(2)

    return candidates

if __name__ == "__main__":
    testcases = input()
      
    for caseNr in xrange(1, testcases+1):
        print("Case #%i:" % caseNr)
        line = raw_input()
        arr = line.split(' ')
        R = int(arr[0])
        N = int(arr[1])
        M = int(arr[2])
        K = int(arr[3])

        # which products can I get
        productToBuildungs = {}
        for r in xrange(0, N+1):
            for product in combinations_with_replacement(range(2,M+1),r):
                s = mul(product)
                if s not in productToBuildungs:
                    productToBuildungs[s] = [1, [list(product)]]
                else:
                    productToBuildungs[s][0] += 1
                    productToBuildungs[s][1].append(list(product))

        for r in xrange(R):
            products = [int(el) for el in raw_input().split(' ') if int(el) != 1]
            print(''.join(map(str, sorted(solve(N, M, products, productToBuildungs)))))

Google Code Jam Templates

info@martin-thoma.de (Martin Thoma) — Fri, 26 Apr 2013 11:17:35 +0200

Here are some templates that are a good start for Google Code Jam.

C++

#include <stdio.h>
#include <string>
#include <iostream>

using namespace std;

int main(void) {
    /* number of test cases */
    unsigned short int testcases;

    cin >> t;

    for(int i=1; i <= t; i++) { //loops for each case
        for (int zeile=0; zeile<4; zeile++) {
            for (int spalte=0; spalte<4; spalte++) {
                cin >> game[zeile][spalte];
            }
        }
        cout << "Case #" << i << ": " << solve() << endl;
    }

    return 0;
}

Compile

g++ A.cpp

Execute

./a.out < A-small-practice.in > result.txt

Python

Input: input, raw_input()
String parsing: strip(), split()

#!/usr/bin/env python
# -*- coding: utf-8 -*-

if __name__ == "__main__":
	testcases = input()
	 
	for caseNr in xrange(1, testcases+1):
		cipher = raw_input()
		print("Case #%i: %s" % (caseNr, solve(cipher)))

Execute

python A.py < A-small-practice.in > result.txt

Java

This is an ajusted version of mystics solution for “Dancing with Googlers”. You might want to take a look at Scanner and PrintWriter.

import java.util.*;
import java.io.*;

public class DancingWithGoogle {
	final static String PROBLEM_NAME = "dance";
	final static String WORK_DIR = "/home/moose/Desktop/" + PROBLEM_NAME + "/";
	final static String INPUT_FILE_NAME = "input.txt";
	final static String OUTPUT_FILE_NAME = "output.txt";

	static int[][] maxBest = new int[31][2];

	static void preprocess() {
		for (int i = 0; i <= 30; i++)
			Arrays.fill(maxBest[i], -1);

		for (int A = 0; A <= 10; A++)
			for (int B = A; B <= 10 && B <= A + 2; B++)
				for (int C = B; C <= 10 && C <= A + 2; C++) {
					int tot = A + B + C, sur = (C - A == 2 ? 1 : 0);
					maxBest[tot][sur] = Math.max(maxBest[tot][sur], C);
				}
	}

	void solve(Scanner sc, PrintWriter pw) {
		int N = sc.nextInt();
		int S = sc.nextInt();
		int p = sc.nextInt();
		int[] tot = new int[N];
		for (int i = 0; i < N; i++)
			tot[i] = sc.nextInt();

		int[][] dp = new int[N + 1][N + 1];
		for (int i = 0; i <= N; i++)
			Arrays.fill(dp[i], -100000);
		dp[0][0] = 0;

		for (int pos = 0; pos < N; pos++)
			for (int sur = 0; sur <= pos; sur++)
				for (int nSur = 0; nSur < 2; nSur++)
					if (maxBest[tot[pos]][nSur] >= 0)
						dp[pos + 1][sur + nSur] = Math.max(dp[pos + 1][sur
								+ nSur], dp[pos][sur]
								+ (maxBest[tot[pos]][nSur] >= p ? 1 : 0));

		pw.println(dp[N][S]);
	}

	public static void main(String[] args) throws Exception {
		preprocess();

		Scanner sc = new Scanner(new FileReader(WORK_DIR + INPUT_FILE_NAME));
		PrintWriter pw = new PrintWriter(new FileWriter(WORK_DIR
				+ OUTPUT_FILE_NAME));
		int caseCnt = sc.nextInt();
		for (int caseNum = 0; caseNum < caseCnt; caseNum++) {
			System.out.println("Processing test case " + (caseNum + 1));
			pw.print("Case #" + (caseNum + 1) + ": ");
			new DancingWithGoogle().solve(sc, pw);
		}
		pw.flush();
		pw.close();
		sc.close();
	}
}

Adjust the path and execute it within Eclipse.

PHP

Input / output:

\`$fp = fopen ($`argv[1], 'r'): Open file pointer to file in first command line argument
string fgets (\$fp): Read one line

Execute:

php A.php input.txt > output.txt

JavaScript

Did you know that you can also solve those tasks with JavaScript? I’ve explained how to install v8.

Here is a solution from aditsu:

// run with v8: d8 file.js < file.in

var m, res, dot

function check(x0, y0, dx, dy) {
	var x = 0
	var o = 0
	var t = 0
	for (var i = 0; i < 4; ++i) {
		switch (m[x0 + i * dx][y0 + i * dy]) {
		case 'X':x++;break
		case 'O':o++;break
		case 'T':t++;break
		case '.':dot=1;break
		}
	}
	if (x + t == 4) res = 'X'
	if (o + t == 4) res = 'O'
}

var t = readline()
for (var i = 1; i <= t; ++i) {
	m = []
	for (var j = 0; j < 4; ++j) {
		m[j] = readline()
	}
	readline()
	res = null
	dot = 0
	for (var j = 0; j < 4; ++j) {
		check(j, 0, 0, 1)
		check(0, j, 1, 0)
	}
	check(0, 0, 1, 1)
	check(3, 0, -1, 1)
	res = res ? res + ' won' : dot ? 'Game has not completed' : 'Draw'
	print('Case #' + i + ': ' + res)
}

Triangle area

info@martin-thoma.de (Martin Thoma) — Thu, 25 Apr 2013 10:13:15 +0200

I’ve just seen the following image on spikedmath.com:

Some geometry questions.
Source: spikedmath.com

The second answers seem to be obviously the correct ones, right? Wrong.

According to Heron’s formula you can calculate a triangles area like this:

Let $a, b, c$ be the side lengths of the triangle. $s := \frac{a+b+c}{2}$ $T = \sqrt{s \cdot (s-a) \cdot (s-b) \cdot (s-c)}$

So the area of the first triangle is $s_1 := \frac{16}{2} = 8$ $T_1 := \sqrt{8 \cdot (8-5) \cdot (8-5) \cdot (8-6)} = \sqrt{8 \cdot 3 \cdot 3 \cdot 2} = 3 \cdot 4 = 12$

The area of the second one is $s_1 := \frac{18}{2} = 9$ $T_1 := \sqrt{9 \cdot (9-5) \cdot (9-5) \cdot (9-8)} = \sqrt{9 \cdot 4 \cdot 4 \cdot 1} = 3 \cdot 4 = 12$

Both triangles have the same area!

When you draw it, it looks like this:

Both triangles in one picture

Rechnernetze-Klausur

info@martin-thoma.de (Martin Thoma) — Thu, 25 Apr 2013 09:32:06 +0200

Dieser Artikel beschäftigt sich mit der Vorlesung „Einführung in Rechnernetze“ des Moduls „Kommunikation und Datenhaltung“ am KIT. Er dient als Prüfungsvorbereitung. Ich habe die Vorlesungen bei Herrn Prof. Dr. Beigl im Sommersemester 2013 gehört.

Behandelter Stoff

16.04.2013	Einleitung
Kapitel 1
24.04.2013	E-Mail, DNS, dig, nslookup, Peer-to-Peer-Netzwerke, Bootstrapping-Problem, ISO-/OSI-Schichtenmodell, SAPs; Dienstgeber/-bringer, -funktionalität, -nehmer, -primitiv, -leistung; Request, Indication, Response, Confirm; QoS
Kapitel 2, Folie 18
30.04.2013	SDU, PDU, ISO / OSI-Architektur
Kapitel 2 bis wohin?
21.05.2013	CRC, Generatorpolynome, Vorwärtsfehlerkorrektur, Sequenznummern, Quittung, Stop-and-wait
?
28.05.2013	Protokollmechanismen und Verbindungen: Stop-and-wait; Go-Back-N ARQ, Flusskontrolle (Open Loop, Closed Loop); Kreditbasierte Flusskontrolle
Kapitel 4.5 - 4.8
04.06.2013	?
Kapitel 5
12.06.2013	Übungsblatt 2: ARQ: Bestätigter Dienst (JA), Zuverlässiger Dienst (JA, wenn...); Stop-and-Wait;
?
18.06.2013	Paketvermittlung
7.2.2
25.06.2013	Broadcast-Routing: Dateneinheit wird für jedes System erstellt; Hot-Potato; Potential des Missbrauchs (→ The Internet could crash. We need a Plan B.); Outlaw-detection; Distanz-Vector-Routing; Link-State-Routing
7-44
02.07.2013	IPv4 (Class A/B/C/D/E-Netze); CIDR; Zuteilung von Adressen (DHCP); Subnetze und Subnetz-Maskierung
7-44

Falls hier was fehlt, könnt ihr mich gerne in den Kommentaren oder per Mail (info@martin-thoma.de) darauf aufmerksam machen. Ich bin ja mal gespannt, ob ich das bis zum Ende aktuell halte.

Übersicht über Dienstprimitive

Name	Dienstleistung	Grundtypen	Parameter
Physical (Ph)	Connect (Con)	Request (Req)	Abhängig vom Dienst
Data Link (DL)	Data (Dat)	Indication (Ind)
Network (N)	Release (Rel)	Response (Rsp)
Transport (T)	Abort (Abo)	Confirmation (Cnf)
HTTP	Provider Abort (PAbo)
FTP	Disconnect (Dis)
...	...

Fragen

Welche Qualitätsparameter sind für Rechnernetze denkbar?

Angemessenheit
Technische Leistung (Antwortzeit, Datenrate)
Zuverlässigkeit
Sicherheit
Kosten

Material

Vorlesungswebsite
Forum
Mein Anki-Deck
Ein Video über CRC
Der Wikipedia-Artikel Routing beinhaltet viele wichtige Informationen.
TCP flags: Hackers Playground

Aufbau der Klausur

Häufige Aufgabenstellungen sind:

Berechnen einer Subnetzmaske bzw. ob eine IP-Adresse in einem gegebenem Subnetz enthalten ist
CRC berechnen / überprüfen ob CRC korrekt ist
Distanz-Vektor Algorithmus (Bellman-Ford) durchgehen
Protokollablauf durchspielen

Übungsbetrieb

Übungsblätter sind hier.

Termine und Klausurablauf

Datum: Donnerstag, den 1. August 2013 von 14:00 bis 15:00 Uhr
Ort: Seit 28.07.2013 online:

Ort	von	bis
Audimax A (30.95, EG)	Abdullah	Galler
Audimax B (30.95, EG)	Gassenschmidt	Löffler
HSaF (50.35, EG)	Loose	Tobias
Neue Chemie (30.46, EG)	Traub	Zumkeller

Einsicht: 17.09.2013
Punkte: 30
Bestehensgrenze: 13 Punkte
Notenskala:

Note	von	bis	Bereichsgröße
1,0	30,00	26,50	3,5
1,3	26,25	25,00	1,25
1,7	24,75	23,50	1,25
2,0	23,25	22,00	1,25
2,3	21,75	20,50	1,25
2,7	20,25	19,00	1,25
3,0	18,75	17,50	1,25
3,3	17,25	16,00	1,25
3,7	15,75	14,50	1,25
4,0	14,25	13,00	1,25
5,0	12,75	00,00	12,75

Übungsschein: Nein
Bonuspunkte: Nein

Nicht vergessen

Studentenausweis
Kugelschreiber

Ergebnisse

Sind nun online. Hier ist die Statistik:

Klausurergebnisse Rechnernetze 2013